Holographic Projection

ABSTRACT

A holographic projector comprises an image processing engine, a hologram engine, a display engine and a light source. The image processing engine is arranged to receive a source image for projection and generate a plurality of secondary images from a primary image based on the source image. The source image comprises pixels. Each secondary image may comprise fewer pixels than the source image. The plurality of secondary images are generated by sampling the primary image. The hologram engine is arranged to determine, such as calculate, a hologram corresponding to each secondary image to form a plurality of holograms. The display engine is arranged to display each hologram on the display device. The light source is arranged to Illuminate each hologram during display to form a holographic reconstruction corresponding to each secondary image on a replay plane. The primary image is selected from the group comprising: the source image and an intermediate image

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of priority of United Kingdom PatentApplication no. 1912168.0, filed August 23, 201, which is herebyincorporated herein by reference in its entirety.

FIELD

The present disclosure relates to an image processor and a projector.More specifically, the present disclosure relates to a holographicprojector, a holographic projection system and an image processor forholographic projection. The present disclosure further relates to amethod of holographically projecting a target image and a method ofholographically projecting video images. Some embodiments relate to ahead-up display.

BACKGROUND AND INTRODUCTION

Light scattered from an object contains both amplitude and phaseinformation. This amplitude and phase information can be captured on,for example, a photosensitive plate by well-known interferencetechniques to form a holographic recording, or “hologram”, comprisinginterference fringes. The hologram may be reconstructed by illuminationwith suitable light to form a two-dimensional or three-dimensionalholographic reconstruction, or replay image, representative of theoriginal object.

Computer-generated holography may numerically simulate the interferenceprocess. A computer-generated hologram, “CGH”, may be calculated by atechnique based on a mathematical transformation such as a Fresnel orFourier transform. These types of holograms may be referred to asFresnel or Fourier holograms. A Fourier hologram may be considered aFourier domain representation of the object or a frequency domainrepresentation of the object. A CGH may also be calculated by coherentray tracing or a point cloud technique, for example.

A CGH may be encoded on a spatial light modulator, “SLM”, arranged tomodulate the amplitude and/or phase of incident light. Light modulationmay be achieved using electrically-addressable liquid crystals,optically-addressable liquid crystals or micro-mirrors, for example.

The SLM may comprise a plurality of individually-addressable pixelswhich may also be referred to as cells or elements. The light modulationscheme may be binary, multilevel or continuous. Alternatively, thedevice may be continuous (i.e. is not comprised of pixels) and lightmodulation may therefore be continuous across the device. The SLM may bereflective meaning that modulated light is output from the SLM inreflection. The SLM may equally be transmissive meaning that modulatedlight is output from the SLM is transmission.

A holographic projector for imaging may be provided using the describedtechnology. Such projectors have found application in head-up displays,“HUD”, and head-mounted displays, “HMD”, including near-eye devices, forexample.

A holographic projector projects an image onto a replay field on areplay plane. When using the described technology, the projected imageis formed from a hologram displayed on pixels of the SLM, hereinreferred to as “SLM pixels”. Thus, the SLM pixels display pixels of thehologram, herein referred to as “hologram pixels”. The projected imageis formed of “image spots” which are also referred to herein as “imagepixels”. The image pixels have a finite size and adjacent image pixelsin the replay field can interfere or blur together. This is referred toherein as pixel crosstalk. The problem of pixel crosstalk leads toreduced image quality.

Furthermore, a hologram engine takes time to determine a hologram fordisplay from a source image. For example, the hologram may be a Fourierhologram calculated using at least one Fourier transform. The time takento calculate the hologram can therefore limit the rate at whichholograms can be written to the SLM and thus the rate at which asequence of source images can be projected as a video stream, hereincalled the “frame rate”. Thus, it can be difficult to project images atacceptable video frame rates.

The present disclosure concerns techniques for implementing timeinterlacing to optimise the resolution of the holographic reconstructionof a source image on the replay plane. Some embodiments disclosed hereininvolve image sampling or sub-sampling of upscaled images and someembodiments involve efficiently compensating for the warping caused byan optical system used to image the holographic reconstruction.

There is disclosed herein an improved holographic projection system andmethod.

SUMMARY

Aspects of the present disclosure are defined in the appendedindependent claims.

There is disclosed herein a holographic projector arranged toholographically-reconstruct a target image by projection. Theholographic projector comprises an image processing engine, a hologramengine, a display engine and a light source. The image processing enginemay be arranged to receive a source image or a target image. The imageprocessing engine may be arranged to upscale the target image to formthe source image. The source image comprises pixels. The imageprocessing engine is arranged to generate a plurality of secondaryimages by sampling a primary image, which is based on the source image.For example, the primary image may be selected from the group comprisingthe source image and an intermediate image derived from the sourceimage. Each secondary image may comprise fewer pixels than the sourceimage. The hologram engine is arranged to determine, such as calculate,a hologram corresponding to each secondary image to form a plurality ofholograms. The display engine is arranged to display each hologram on adisplay device. Each hologram may be displayed in turn on a displaydevice. Alternatively, or additionally, two or more display devices ortwo or more zones or areas within the same display device may beprovided, to display two or more respective holograms substantiallyconcurrently. The light source is arranged to Illuminate each hologramduring display to form a holographic reconstruction corresponding toeach secondary image on a replay plane.

The upscaling of a target image to form the source image may compriserepeating each pixel value of the target image in respective contiguousgroup of pixels of the source image, wherein there is a positionalcorrespondence between each pixel of the target image and thecorresponding group of pixels of the source image having the same pixelvalue.

Each secondary image may comprise a plurality of pixels, calculated fromcorresponding groups of pixels of the primary image at a plurality ofpositions of a sampling window. Each pixel value of each secondary imagemay be calculated from a corresponding group that comprises a pluralityof pixels of the primary image that fall within the sampling window at arespective one of the plurality of sampling window positions. Theplurality of positions of a sampling window, for generating a specificsecondary image, may comprise a checkerboard pattern, with each samplingwindow position being separated from its nearest-neighbour samplingwindow position, in each of the x and y directions. Alternatively, theplurality of positions of a sampling window, for generating a specificsecondary image, may be contiguous (i.e. directly adjacent andnon-overlapping) with one another. The plurality of positions of asampling window, for generating a first secondary image, may bedifferent to the plurality of positions of the sampling window, forgenerating a second, different secondary image of the same source imageor a target image. For example, the checkerboard pattern used forgenerating a first secondary image may be the opposite of thecheckerboard pattern used for generating a second secondary image. Forexample, the plurality of sampling window positions used for generatinga second secondary image may be offset, such as diagonally offset, froma plurality of positions used for generating a first secondary image.The same size and shape of sampling window may be used, for generatingthe pixel values for every pixel of a secondary image. The same size andshape of sampling window may be used, for generating the pixel valuesfor each pixel of both a first secondary image and a second secondaryimage of the same source image or target image.

The inventors have disclosed herein an approach in which a plurality ofa secondary images is derived by sampling a primary image. The primaryimage may correspond to the source image or an image derived from thesource image (herein an “intermediate image”). A hologram is determinedand displayed for each secondary image. A corresponding plurality ofholographic reconstructions are therefore formed on the replay plane,either concurrently or one after the other. The holographicreconstructions are formed within the integration time of the human eyesuch that a viewer cannot tell that the projected image that they see isformed either from multiple holographic reconstructions, formed oneafter the other, and/or from holograms that are displayed on multiplerespective display areas or display devices. The projected imagetherefore appears to be a faithful and complete reconstruction of thesource image. Due to the technique of sampling, each secondary image canhave lower resolution than the primary image. Provided the primary imagehas a sufficiently high resolution, a desired resolution of the completeholographic reconstruction of the source image can be achieved.

According to an aspect, a holographic projector is provided, wherein theholographic projector is arranged to project a target image. Theholographic projector comprises an image processing engine arranged togenerate a plurality of secondary images by sampling a primary imagederived from the target image, wherein each secondary image may comprisefewer pixels than the primary image. The holographic projector furthercomprises a hologram engine arranged to determine a hologramcorresponding to each secondary image to form a plurality of holograms,and a display engine arranged to display each hologram on a displaydevice. The holograms may be displayed in turn on the same displaydevice and/or on different respective display devices and/or ondifferent respective zones or areas of a common display device. Theholographic projector further comprises a light source arranged toIlluminate each hologram during display to form a holographicreconstruction corresponding to each secondary image on a replay plane.Each secondary image may comprise a plurality of pixels, calculated fromcorresponding groups of pixels of the primary image at a plurality ofpositions of a sampling window, wherein each pixel value of secondaryimage may be calculated from a corresponding group that comprises aplurality of pixels of the primary image that fall within the samplingwindow at a respective one of the plurality of sampling windowpositions. At least some of the plurality of positions of the samplingwindow that are used for generating a first secondary image may bedifferent to at least some of a plurality of positions of the samplingwindow that are used for generating a second different secondary image.

According to an aspect, a holographic projector is provided, wherein theholographic projector is arranged to project a target image. Theholographic projector comprises an image processing engine arranged togenerate a plurality of secondary images by sampling a primary imagederived from the target image, wherein each secondary image may comprisefewer pixels than the primary image. The holographic projector furthercomprises a hologram engine arranged to determine a hologramcorresponding to each secondary image, to form a plurality of holograms,and a display engine arranged to display each hologram on one or moredisplay devices. The holograms may be displayed in turn on the samedisplay device and/or on different respective display devices and/or ondifferent respective zones or areas of a common display device. Theholographic projector further comprises a light source arranged toIlluminate each hologram during display to form a holographicreconstruction corresponding to each secondary image on a replay plane.Each secondary image comprises a plurality of pixels, each of which maybe calculated from corresponding groups of pixels of the primary imageat a plurality of respective positions of a sampling window.

According to this aspect, the sampling may comprise calculating thepixel value of each pixel of a secondary image by individually weightingthe pixel values of a respective group of pixels of the primary image,which fall within the sampling window at a respective one of itsplurality of positions, such that there is a positional correspondencebetween each pixel of said secondary image and the respective group ofpixels of the primary image. A first set of sampling window positionsmay be used for calculating a first secondary image and a second,different set of sampling window positions may be used for calculating arespective second secondary image, within the plurality of secondaryimages. The first set of sampling window positions may be offset, forexample diagonally offset, from the second set of sampling windowpositions. A first holographic reconstruction may be formed bydisplaying and illuminating a hologram corresponding to the firstsecondary image, spatially-displaced on the replay plane relative to asecond holographic reconstruction formed by displaying and illuminatinga hologram corresponding to the second secondary image, in order tointerlace the first and second holographic reconstructions.

According to an aspect, a holographic projector is provided, wherein theholographic projector is arranged to project a target image, theholographic projector comprising an image processing engine arranged togenerate a plurality of secondary images by sampling a primary imagederived from the target image, wherein each secondary image may comprisefewer pixels than the primary image. The primary image may comprise ormay be derived from a source image, which comprises an upscaled versionof the target image. The holographic projector may further comprise ahologram engine arranged to determine a hologram corresponding to eachsecondary image to form a plurality of holograms, and a display enginearranged to display each hologram on a display device. The holograms maybe displayed in turn on the same display device and/or on differentrespective display devices and/or on different respective zones or areasof a common display device. The holographic projector further comprisesa light source arranged to Illuminate each hologram during display toform a holographic reconstruction corresponding to each secondary imageon a replay plane.

According to this aspect, each secondary image may comprise a pluralityof pixels, calculated from corresponding groups of pixels of the primaryimage at a plurality of positions of a sampling window, wherein thesampling comprises calculating the pixel value of each pixel of asecondary image from a respective group of pixels of the primary image,which fall within the sampling window at a respective one of itsplurality of positions, such that there is a positional correspondencebetween each pixel of said secondary image and the respective group ofpixels of the primary image. There may be a first ratio between theresolution of the target image and the resolution of the source imageand a second, different ratio between the resolution of each one of thesecondary images and the resolution of the source image. In other words;there may be a first ratio between the number of pixels in a hologramcorresponding to the target image and the number of pixels in a hologramcorresponding to the source image and a second, different ratio betweenthe number of pixels in each respective hologram corresponding to eachone of the secondary images and the number of pixels in a hologramcorresponding to the source image. Therefore, a desired or required netratio, between the resolution of the target image and the resolution ofone or more secondary images, may be achieved.

The approaches described herein provide significant technicalcontributions to the field. Firstly, the quality of the projected imageis improved. Secondly, the speed at which the projected image can beupdated (i.e., the frame rate) is increased. Thirdly, a more flexibleholographic projector is provided. These technical contributions areexplained respectively in the following paragraphs.

First, the approach disclosed herein enables pixel crosstalk to bemanaged by displaying different image pixels at different times or ondifferent respective display devices (or in different respective zonesor sections of a common display device). More specifically, differentgroups of image spots are displayed at different times or on differentrespective display devices (or in different respective zones or sectionsof a common display device). For example, a first holographicreconstruction formed at a first time (corresponding to a firstsecondary image) may comprise a first group of image pixels (e.g., everyother image pixel or pixels formed from sampling a primary image at afirst plurality of sampling window positions) of an image frame and asecond holographic reconstruction at a second time (corresponding to asecond secondary image) may fill in the gaps of the image frame bydisplaying a second group comprising the remaining image pixels (orpixels formed from sampling the primary image at a second, differentplurality of sampling window positions. Since image pixels of the firstand second groups (e.g. adjacent pixel groups) are not displayed at thesame time, interpixel interference and pixel crosstalk is reduced. Theinventors have therefore disclosed a technique of interlacing (in timeand/or in space) a plurality of holographic reconstructions to improveimage quality by managing pixel crosstalk.

In the present disclosure, the new approaches are implemented bysampling a high-resolution source image in a plurality of different waysto obtain a respective plurality of secondary images. Thus, it ispossible to achieve a desired resolution of the interlaced holographicreconstructions by “upscaling” the target image to form a source image,and sampling the source image or an intermediate image based on thesource image, whilst managing pixel crosstalk.

Secondly, the inventors have disclosed herein approaches which aresuitable for real-time (i.e. video rate) processing. Specifically, theholograms can be determined and displayed within the frame time ofvideo. This technical contribution is achieved because each secondaryimage may have fewer pixels than the source image. Although moreholograms are required for reconstructing each source image, when thesecondary images have fewer pixels than the source image, eachindividual hologram can be determined much more quickly. For example, itis quicker to calculate two holograms comprising x pixels using aFourier transform method than it is to calculate one hologram comprising2x pixels. The inventors have therefore disclosed a technique toincrease the speed of calculating holograms corresponding to a sourceimage to enable holographic projection at acceptable video frame rates.

These and other advantages of the new approach disclosed herein will befurther appreciated from the following detailed description.

The term “target image” is used herein to reference to the input to theholographic system described herein. That is, the target image is theimage that the holographic system is required to project onto aholographic replay plane. The target image may be one image of asequence of images such as a video-rate sequence of images.

The term “source image” is used herein to refer to an image derived fromthe target image. The source image may be the same as the target imageor the source image may be an upscaled version of the target image. Thatis, the source image may comprise more pixels than the target image. Anyupscaling technique may be employed. In some embodiments, upscalingcomprises repeating pixel values of the target image, as described inthe detailed description. In these embodiments, the computational enginemay use a simple mapping scheme to represent the repeating.

The term “warping” is used herein to refer to the process by which animage is distorted by the optics of an optical system, such as anoptical relay system, used to image the holographic reconstruction. Theoptical system may include elements having non-uniform optical power. A“warping map” is a mathematical function or mapping scheme whichdescribes/defines how an image will be changed (e.g. distorted) by theoptical system. Specifically, warping maps describe how discrete points(e.g. pixels or pixel areas) of an image will be changed (e.g.displaced/translated) by the optical system. The holographic systemdisclosed herein may anticipate/model/predict the warping that willoccur using the warping maps. Some techniques disclosed herein require awarping map (or pair of warping maps—e.g. x- and y-warping maps) but theprocess by which the warping map/s are determined/calculated is notrelevant to the inventions disclosed herein—but examples are brieflyoutlined to help the reader. In examples in which the optical systemimages each holographic reconstruction and each image is visible withinan eye-box region, a warping map pair may be defined for a plurality ofeye-box positions.

The term “primary image” is used herein to refer to either (1) thesource image or (2) an intermediate image derived from the source image.In the description of embodiments, the term “intermediate image” is usedherein to refer to an image derived from the source image in accordancewith a warping map. Specifically, the term “intermediate image” is usedherein to refer to a warped version of the source image—that is, thesource image after warping using a warping map or pair of warping maps,wherein the warping map/s characterise the distortion caused by acorresponding optical system.

The term “secondary image” is used herein to refer to one of a pluralityof images derived from the primary image. Each secondary image is formedby sub-sampling (also referred to as “sampling” and which may bereferred to as “under-sampling”) the primary image. Each secondary imagemay comprise fewer pixels than the source image. Each pixel value of thesecondary image may be calculated from several pixel values of theprimary image, optionally, using a weighting technique as described inthe detailed description. Notably, the upscaling process used to formthe source image from the target image is different to the sub-samplingtechnique used to form each secondary image from the primary image. Thesecondary images are each different to the primary image but,optionally, they may have the same number of pixels or more pixels thanthe primary image. If the secondary images have fewer pixels than thesource image, the pixels of the secondary images can comprisecontributions from each of the pixels of the source image. A hologramcorresponding to each secondary image is calculated.

The term “hologram” is used to refer to the recording which containsamplitude information or phase information, or some combination thereof,about the object. The term “holographic reconstruction” is used to referto the optical reconstruction of the object which is formed byilluminating the hologram. The term “replay plane” is used herein torefer to the plane in space where the holographic reconstruction isfully formed. The term “replay field” is used herein to refer to thesub-area of the replay plane which can receive spatially-modulated lightfrom the spatial light modulator. The terms “image”, “replay image” and“image region” refer to areas of the replay field illuminated by lightforming the holographic reconstruction. In embodiments, the “image” maycomprise discrete spots which may be referred to as “image pixels”.

The terms “encoding”, “writing” or “addressing” are used to describe theprocess of providing the plurality of pixels of the SLM with a respectplurality of control values which respectively determine the modulationlevel of each pixel. It may be said that the pixels of the SLM areconfigured to “display” a light modulation distribution in response toreceiving the plurality of control values. Thus, the SLM may be said to“display” a hologram.

It has been found that a holographic reconstruction of acceptablequality can be formed from a “hologram” containing only phaseinformation related to the original object. Such a holographic recordingmay be referred to as a phase-only hologram. Embodiments relate to aphase-only hologram but the present disclosure is equally applicable toamplitude-only holography.

The present disclosure is also equally applicable to forming aholographic reconstruction using amplitude and phase information relatedto the original object. In some embodiments, this is achieved by complexmodulation using a so-called fully complex hologram which contains bothamplitude and phase information related to the original object. Such ahologram may be referred to as a fully-complex hologram because thevalue (grey level) assigned to each pixel of the hologram has anamplitude and phase component. The value (grey level) assigned to eachpixel may be represented as a complex number having both amplitude andphase components. In some embodiments, a fully-complexcomputer-generated hologram is calculated.

Reference may be made to the phase value, phase component, phaseinformation or, simply, phase of pixels of the computer-generatedhologram or the spatial light modulator as shorthand for “phase-delay”.That is, any phase value described is, in fact, a number (e.g. in therange 0 to 2π) which represents the amount of phase retardation providedby that pixel. For example, a pixel of the spatial light modulatordescribed as having a phase value of π/2 will change the phase ofreceived light by π/2 radians. In some embodiments, each pixel of thespatial light modulator is operable in one of a plurality of possiblemodulation values (e.g. phase delay values). The term “grey level” maybe used to refer to the plurality of available modulation levels. Forexample, the term “grey level” may be used for convenience to refer tothe plurality of available phase levels in a phase-only modulator eventhough different phase levels do not provide different shades of grey.The term “grey level” may also be used for convenience to refer to theplurality of available complex modulation levels in a complex modulator.

Although different examples and embodiments may be disclosed separatelyin the detailed description which follows, any feature of any example orembodiment may be combined with any other feature or combination offeatures of any example or embodiment. That is, all possiblecombinations and permutations of features disclosed in the presentdisclosure are envisaged.

BRIEF DESCRIPTION OF THE DRAWINGS

Specific embodiments are described by way of example only with referenceto the following figures:

FIG. 1 is a schematic showing a reflective SLM producing a holographicreconstruction on a screen;

FIG. 2A illustrates a first iteration of an example Gerchberg-Saxtontype algorithm;

FIG. 2B illustrates the second and subsequent iterations of the exampleGerchberg-Saxton type algorithm;

FIG. 2C illustrates alternative second and subsequent iterations of theexample Gerchberg-Saxton type algorithm;

FIG. 3 is a schematic of a reflective LCOS SLM;

FIG. 4 shows an example technique for determining a pair of hologramsfrom respective secondary images derived from a source image forprojection by a holographic projector in accordance with embodiments;

FIG. 5 shows holographic reconstructions produced by sequentiallydisplaying a pair of holograms, based on a generalisation of the exampletechnique of FIG. 4, in accordance with embodiments;

FIG. 6 shows an example source image;

FIG. 7A shows an example technique for sampling the source image of FIG.6 to derive a first secondary image in accordance with embodiments;

FIG. 7B shows an example technique for sampling the source image of FIG.6 to derive a second secondary image that is diagonally offset to thefirst secondary image in accordance with embodiments;

FIG. 8 shows example kernels for use in the sampling techniques of FIGS.7A and 7B;

FIG. 9 shows diagonally offset first and second time interlacedholographic reconstructions formed by display of first and secondholograms determined for the respective first and second secondaryimages of FIGS. 7A and 7B in accordance with embodiments;

FIG. 10A shows an example source image;

FIG. 10B shows an example warped image, referred to as an intermediateimage;

FIG. 11A shows sampling of the intermediate image of FIG. 10B todetermine a first secondary image and FIG. 11B shows the first secondaryimage, in accordance with embodiments;

FIG. 12 shows a magnified view of a part of FIG. 11A;

FIG. 13A shows sampling of the intermediate image of FIG. 10B todetermine a second secondary image, and FIG. 13B shows the secondsecondary image, in accordance with embodiments;

FIG. 14 illustrates an example displacement map;

FIG. 15 illustrates another example displacement map;

FIG. 16A shows a target image for projection and FIG. 16B shows anupscaled version of the target image in accordance with someembodiments;

FIG. 17A shows a checkerboarding approach to sub-sampling the upscaledtarget image;

FIGS. 17B and 17C show how the different areas of the checkerboard arewarped by optics of an optical system which images the holographicreplay field;

FIG. 18 shows a sampling window for sub-sampling in accordance with someembodiments;

FIG. 19 is a schematic showing a holographic projector in accordancewith embodiments.

The same reference numbers will be used throughout the drawings to referto the same or like parts.

DETAILED DESCRIPTION OF EMBODIMENTS

The present invention is not restricted to the embodiments described inthe following but extends to the full scope of the appended claims. Thatis, the present invention may be embodied in different forms and shouldnot be construed as limited to the described embodiments, which are setout for the purpose of illustration.

Terms of a singular form may include plural forms unless specifiedotherwise.

A structure described as being formed at an upper portion/lower portionof another structure or on/under the other structure should be construedas including a case where the structures contact each other and,moreover, a case where a third structure is disposed there between.

In describing a time relationship—for example, when the temporal orderof events is described as “after”, “subsequent”, “next”, “before” orsuchlike—the present disclosure should be taken to include continuousand non-continuous events unless otherwise specified. For example, thedescription should be taken to include a case which is not continuousunless wording such as “just”, “immediate” or “direct” is used.

Although the terms “first”, “second”, etc. may be used herein todescribe various elements, these elements are not be limited by theseterms. These terms are only used to distinguish one element fromanother. For example, a first element could be termed a second element,and, similarly, a second element could be termed a first element,without departing from the scope of the appended claims.

Features of different embodiments may be partially or overall coupled toor combined with each other, and may be variously inter-operated witheach other. Some embodiments may be carried out independently from eachother, or may be carried out together in co-dependent relationship.

Optical Configuration

FIG. 1 shows an embodiment in which a computer-generated hologram isencoded on a single spatial light modulator. The computer-generatedhologram is a Fourier transform of the object for reconstruction. It maytherefore be said that the hologram is a Fourier domain or frequencydomain or spectral domain representation of the object. In thisembodiment, the spatial light modulator is a reflective liquid crystalon silicon, “LCOS”, device. The hologram is encoded on the spatial lightmodulator and a holographic reconstruction is formed at a replay field,for example, a light receiving surface such as a screen or diffuser.

A light source 110, for example a laser or laser diode, is disposed toilluminate the SLM 140 via a collimating lens 111. The collimating lenscauses a generally planar wavefront of light to be incident on the SLM.In FIG. 1, the direction of the wavefront is off-normal (e.g. two orthree degrees away from being truly orthogonal to the plane of thetransparent layer). However, in other embodiments, the generally planarwavefront is provided at normal incidence and a beam splitterarrangement is used to separate the input and output optical paths. Inthe embodiment shown in FIG. 1, the arrangement is such that light fromthe light source is reflected off a mirrored rear surface of the SLM andinteracts with a light-modulating layer to form an exit wavefront 112.The exit wavefront 112 is applied to optics including a Fouriertransform lens 120, having its focus at a screen 125. More specifically,the Fourier transform lens 120 receives a beam of modulated light fromthe SLM 140 and performs a frequency-space transformation to produce aholographic reconstruction at the screen 125.

Notably, in this type of holography, each pixel of the hologramcontributes to the whole reconstruction. There is not a one-to-onecorrelation between specific points (or image pixels) on the replayfield and specific light-modulating elements (or hologram pixels). Inother words, modulated light exiting the light-modulating layer isdistributed across the replay field.

In these embodiments, the position of the holographic reconstruction inspace is determined by the dioptric (focusing) power of the Fouriertransform lens. In the embodiment shown in FIG. 1, the Fourier transformlens is a physical lens. That is, the Fourier transform lens is anoptical Fourier transform lens and the Fourier transform is performedoptically. Any lens can act as a Fourier transform lens but theperformance of the lens will limit the accuracy of the Fourier transformit performs. The skilled person understands how to use a lens to performan optical Fourier transform.

Hologram Calculation

In some embodiments, the computer-generated hologram is a Fouriertransform hologram, or simply a Fourier hologram or Fourier-basedhologram, in which an image is reconstructed in the far field byutilising the Fourier transforming properties of a positive lens. TheFourier hologram is calculated by Fourier transforming the desired lightfield in the replay plane back to the lens plane. Computer-generatedFourier holograms may be calculated using Fourier transforms.

A Fourier transform hologram may be calculated using an algorithm suchas the Gerchberg-Saxton algorithm. Furthermore, the Gerchberg-Saxtonalgorithm may be used to calculate a hologram in the Fourier domain(i.e. a Fourier transform hologram) from amplitude-only information inthe spatial domain (such as a photograph). The phase information relatedto the object is effectively “retrieved” from the amplitude-onlyinformation in the spatial domain. In some embodiments, acomputer-generated hologram is calculated from amplitude-onlyinformation using the Gerchberg-Saxton algorithm or a variation thereof.

The Gerchberg Saxton algorithm considers the situation when intensitycross-sections of a light beam, I_(A)(x, y) and I_(B)(x, y), in theplanes A and B respectively, are known and I_(A)(x, y) and I_(B)(x, y)are related by a single Fourier transform. With the given intensitycross-sections, an approximation to the phase distribution in the planesA and B, ψ_(A)(x, y) and ψ_(B)(x, y) respectively, is found. TheGerchberg-Saxton algorithm finds solutions to this problem by followingan iterative process. More specifically, the Gerchberg-Saxton algorithmiteratively applies spatial and spectral constraints while repeatedlytransferring a data set (amplitude and phase), representative ofI_(A)(x, y) and I_(B)(x, y), between the spatial domain and the Fourier(spectral or frequency) domain. The corresponding computer-generatedhologram in the spectral domain is obtained through at least oneiteration of the algorithm. The algorithm is convergent and arranged toproduce a hologram representing an input image. The hologram may be anamplitude-only hologram, a phase-only hologram or a fully complexhologram.

In some embodiments, a phase-only hologram is calculated using analgorithm based on the Gerchberg-Saxton algorithm such as described inBritish patent 2,498,170 or 2,501,112 which are hereby incorporated intheir entirety by reference. However, embodiments disclosed hereindescribe calculating a phase-only hologram by way of example only. Inthese embodiments, the Gerchberg-Saxton algorithm retrieves the phaseinformation ψ[u, v] of the Fourier transform of the data set which givesrise to a known amplitude information T[x, y], wherein the amplitudeinformation T[x, y] is representative of a target image (e.g. aphotograph). Since the magnitude and phase are intrinsically combined inthe Fourier transform, the transformed magnitude and phase containuseful information about the accuracy of the calculated data set. Thus,the algorithm may be used iteratively with feedback on both theamplitude and the phase information. However, in these embodiments, onlythe phase information ψ[u, v] is used as the hologram to form aholographic representative of the target image at an image plane. Thehologram is a data set (e.g. 2D array) of phase values.

In other embodiments, an algorithm based on the Gerchberg-Saxtonalgorithm is used to calculate a fully-complex hologram. A fully-complexhologram is a hologram having a magnitude component and a phasecomponent. The hologram is a data set (e.g. 2D array) comprising anarray of complex data values wherein each complex data value comprises amagnitude component and a phase component.

In some embodiments, the algorithm processes complex data and theFourier transforms are complex Fourier transforms. Complex data may beconsidered as comprising (i) a real component and an imaginary componentor (ii) a magnitude component and a phase component. In someembodiments, the two components of the complex data are processeddifferently at various stages of the algorithm.

FIG. 2A illustrates the first iteration of an algorithm in accordancewith some embodiments for calculating a phase-only hologram. The inputto the algorithm is an input image 210 comprising a 2D array of pixelsor data values, wherein each pixel or data value is a magnitude, oramplitude, value. That is, each pixel or data value of the input image210 does not have a phase component. The input image 210 may thereforebe considered a magnitude-only or amplitude-only or intensity-onlydistribution. An example of such an input image 210 is a photograph orone frame of video comprising a temporal sequence of frames. The firstiteration of the algorithm starts with a data forming step 202Acomprising assigning a random phase value to each pixel of the inputimage, using a random phase distribution (or random phase seed) 230, toform a starting complex data set wherein each data element of the setcomprising magnitude and phase. It may be said that the starting complexdata set is representative of the input image in the spatial domain.

First processing block 250 receives the starting complex data set andperforms a complex Fourier transform to form a Fourier transformedcomplex data set. Second processing block 253 receives the Fouriertransformed complex data set and outputs a hologram 280A. In someembodiments, the hologram 280A is a phase-only hologram. In theseembodiments, second processing block 253 quantiles each phase value andsets each amplitude value to unity in order to form hologram 280A. Eachphase value is quantised in accordance with the phase-levels which maybe represented on the pixels of the spatial light modulator which willbe used to “display” the phase-only hologram. For example, if each pixelof the spatial light modulator provides 256 different phase levels, eachphase value of the hologram is quantised into one phase level of the 256possible phase levels. Hologram 280A is a phase-only Fourier hologramwhich is representative of an input image. In other embodiments, thehologram 280A is a fully complex hologram comprising an array of complexdata values (each including an amplitude component and a phasecomponent) derived from the received Fourier transformed complex dataset. In some embodiments, second processing block 253 constrains eachcomplex data value to one of a plurality of allowable complex modulationlevels to form hologram 280A. The step of constraining may includesetting each complex data value to the nearest allowable complexmodulation level in the complex plane. It may be said that hologram 280Ais representative of the input image in the spectral or Fourier orfrequency domain. In some embodiments, the algorithm stops at thispoint.

However, in other embodiments, the algorithm continues as represented bythe dotted arrow in FIG. 2A. In other words, the steps which follow thedotted arrow in FIG. 2A are optional (i.e. not essential to allembodiments).

Third processing block 256 receives the modified complex data set fromthe second processing block 253 and performs an inverse Fouriertransform to form an inverse Fourier transformed complex data set. Itmay be said that the inverse Fourier transformed complex data set isrepresentative of the input image in the spatial domain.

Fourth processing block 259 receives the inverse Fourier transformedcomplex data set and extracts the distribution of magnitude values 211Aand the distribution of phase values 213A. Optionally, the fourthprocessing block 259 assesses the distribution of magnitude values 211A.Specifically, the fourth processing block 259 may compare thedistribution of magnitude values 211A of the inverse Fourier transformedcomplex data set with the input image 510 which is itself, of course, adistribution of magnitude values. If the difference between thedistribution of magnitude values 211A and the input image 210 issufficiently small, the fourth processing block 259 may determine thatthe hologram 280A is acceptable.

That is, if the difference between the distribution of magnitude values211A and the input image 210 is sufficiently small, the fourthprocessing block 259 may determine that the hologram 280A is asufficiently-accurate representative of the input image 210. In someembodiments, the distribution of phase values 213A of the inverseFourier transformed complex data set is ignored for the purpose of thecomparison. It will be appreciated that any number of different methodsfor comparing the distribution of magnitude values 211A and the inputimage 210 may be employed and the present disclosure is not limited toany particular method. In some embodiments, a mean square difference iscalculated and if the mean square difference is less than a thresholdvalue, the hologram 280A is deemed acceptable. If the fourth processingblock 259 determines that the hologram 280A is not acceptable, a furtheriteration of the algorithm may be performed. However, this comparisonstep is not essential and in other embodiments, the number of iterationsof the algorithm performed is predetermined or preset or user-defined.

FIG. 2B represents a second iteration of the algorithm and any furtheriterations of the algorithm. The distribution of phase values 213A ofthe preceding iteration is fed-back through the processing blocks of thealgorithm. The distribution of magnitude values 211A is rejected infavour of the distribution of magnitude values of the input image 210.In the first iteration, the data forming step 202A formed the firstcomplex data set by combining distribution of magnitude values of theinput image 210 with a random phase distribution 230. However, in thesecond and subsequent iterations, the data forming step 202B comprisesforming a complex data set by combining (i) the distribution of phasevalues 213A from the previous iteration of the algorithm with (ii) thedistribution of magnitude values of the input image 210.

The complex data set formed by the data forming step 202B of FIG. 2B isthen processed in the same way described with reference to FIG. 2A toform second iteration hologram 280B. The explanation of the process isnot therefore repeated here. The algorithm may stop when the seconditeration hologram 280B has been calculated. However, any number offurther iterations of the algorithm may be performed. It will beunderstood that the third processing block 256 is only required if thefourth processing block 259 is required or a further iteration isrequired. The output hologram 280B generally gets better with eachiteration. However, in practice, a point is usually reached at which nomeasurable improvement is observed or the positive benefit of performinga further iteration is out-weighted by the negative effect of additionalprocessing time. Hence, the algorithm is described as iterative andconvergent.

FIG. 2C represents an alternative embodiment of the second andsubsequent iterations. The distribution of phase values 213A of thepreceding iteration is fed-back through the processing blocks of thealgorithm. The distribution of magnitude values 211A is rejected infavour of an alternative distribution of magnitude values. In thisalternative embodiment, the alternative distribution of magnitude valuesis derived from the distribution of magnitude values 211 of the previousiteration. Specifically, processing block 258 subtracts the distributionof magnitude values of the input image 210 from the distribution ofmagnitude values 211 of the previous iteration, scales that differenceby a gain factor α and subtracts the scaled difference from the inputimage 210. This is expressed mathematically by the following equations,wherein the subscript text and numbers indicate the iteration number:

R _(n+1)[x,y]=F′{exp(iψ _(n)[u,v])}

ψ_(n)[u,v]=∠F{η·exp(i∠R _(n)[x,y])}

η=T[x,y]−α(|R _(n)[x,y]|−T[x,y])

where:

F′ is the inverse Fourier transform;

F is the forward Fourier transform;

R[x, y] is the complex data set output by the third processing block256;

T[x, y] is the input or target image;

∠ is the phase component;

ψ is the phase-only hologram 280B;

η is the new distribution of magnitude values 211B; and

α is the gain factor.

The gain factor α may be fixed or variable. In some embodiments, thegain factor α is determined based on the size and rate of the incomingtarget image data. In some embodiments, the gain factor α is dependenton the iteration number. In some embodiments, the gain factor α issolely function of the iteration number.

The embodiment of FIG. 2C is the same as that of FIG. 2A and FIG. 2B inall other respects. It may be said that the phase-only hologram ψ(u, v)comprises a phase distribution in the frequency or Fourier domain.

In some embodiments, the Fourier transform is performed computationallyby including lensing data in the holographic data. That is, the hologramincludes data representative of a lens as well as data representing theobject. In these embodiments, the physical Fourier transform lens 120 ofFIG. 1 is omitted. It is known in the field of computer-generatedhologram how to calculate holographic data representative of a lens. Theholographic data representative of a lens may be referred to as asoftware lens. For example, a phase-only holographic lens may be formedby calculating the phase delay caused by each point of the lens owing toits refractive index and spatially-variant optical path length. Forexample, the optical path length at the centre of a convex lens isgreater than the optical path length at the edges of the lens. Anamplitude-only holographic lens may be formed by a Fresnel zone plate.It is also known in the art of computer-generated hologram how tocombine holographic data representative of a lens with holographic datarepresentative of the object so that a Fourier transform can beperformed without the need for a physical Fourier lens. In someembodiments, lensing data is combined with the holographic data bysimple addition such as simple vector addition. In some embodiments, aphysical lens is used in conjunction with a software lens to perform theFourier transform. Alternatively, in other embodiments, the Fouriertransform lens is omitted altogether such that the holographicreconstruction takes place in the far-field. In further embodiments, thehologram may include grating data—that is, data arranged to perform thefunction of a grating such as beam steering. Again, it is known in thefield of computer-generated holography how to calculate such holographicdata and combine it with holographic data representative of the object.For example, a phase-only holographic grating may be formed by modellingthe phase delay caused by each point on the surface of a blazed grating.An amplitude-only holographic grating may be simply superimposed on anamplitude-only hologram representative of an object to provide angularsteering of an amplitude-only hologram.

In some embodiments, the Fourier transform is performed jointly by aphysical Fourier transform lens and a software lens. That is, someoptical power which contributes to the Fourier transform is provided bya software lens and the rest of the optical power which contributes tothe Fourier transform is provided by a physical optic or optics.

In some embodiments, there is provided a real-time engine arranged toreceive image data and calculate holograms in real-time using thealgorithm. In some embodiments, the image data is a video comprising asequence of image frames. In other embodiments, the holograms arepre-calculated, stored in computer memory and recalled as needed fordisplay on a SLM. That is, in some embodiments, there is provided arepository of predetermined holograms.

Embodiments relate to Fourier holography and Gerchberg-Saxton typealgorithms by way of example only. The present disclosure is equallyapplicable to Fresnel holography and holograms calculated by othertechniques such as those based on point cloud methods.

Light Modulation

A spatial light modulator may be used to display the computer-generatedhologram. If the hologram is a phase-only hologram, a spatial lightmodulator which modulates phase is required. If the hologram is afully-complex hologram, a spatial light modulator which modulates phaseand amplitude may be used or a first spatial light modulator whichmodulates phase and a second spatial light modulator which modulatesamplitude may be used.

In some embodiments, the light-modulating elements (i.e. the pixels) ofthe spatial light modulator are cells containing liquid crystal. Thatis, in some embodiments, the spatial light modulator is a liquid crystaldevice in which the optically-active component is the liquid crystal.Each liquid crystal cell is configured to selectively-provide aplurality of light modulation levels. That is, each liquid crystal cellis configured at any one time to operate at one light modulation levelselected from a plurality of possible light modulation levels. Eachliquid crystal cell is dynamically-reconfigurable to a different lightmodulation level from the plurality of light modulation levels. In someembodiments, the spatial light modulator is a reflective liquid crystalon silicon (LCOS) spatial light modulator but the present disclosure isnot restricted to this type of spatial light modulator.

A LCOS device provides a dense array of light modulating elements, orpixels, within a small aperture (e.g. a few centimeters in width). Thepixels are typically approximately 10 microns or less which results in adiffraction angle of a few degrees meaning that the optical system canbe compact. It is easier to adequately illuminate the small aperture ofa LCOS SLM than it is the larger aperture of other liquid crystaldevices. An LCOS device is typically reflective which means that thecircuitry which drives the pixels of a LCOS SLM can be buried under thereflective surface. The results in a higher aperture ratio. In otherwords, the pixels are closely packed meaning there is very little deadspace between the pixels. This is advantageous because it reduces theoptical noise in the replay field. A LCOS SLM uses a silicon backplanewhich has the advantage that the pixels are optically flat. This isparticularly important for a phase modulating device.

A suitable LCOS SLM is described below, by way of example only, withreference to FIG. 3. An LCOS device is formed using a single crystalsilicon substrate 302. It has a 2D array of square planar aluminiumelectrodes 301, spaced apart by a gap 301 a, arranged on the uppersurface of the substrate. Each of the electrodes 301 can be addressedvia circuitry 302 a buried in the substrate 302. Each of the electrodesforms a respective planar mirror. An alignment layer 303 is disposed onthe array of electrodes, and a liquid crystal layer 304 is disposed onthe alignment layer 303. A second alignment layer 305 is disposed on theplanar transparent layer 306, e.g. of glass. A single transparentelectrode 307 e.g. of ITO is disposed between the transparent layer 306and the second alignment layer 305.

Each of the square electrodes 301 defines, together with the overlyingregion of the transparent electrode 307 and the intervening liquidcrystal material, a controllable phase-modulating element 308, oftenreferred to as a pixel. The effective pixel area, or fill factor, is thepercentage of the total pixel which is optically active, taking intoaccount the space between pixels 301 a. By control of the voltageapplied to each electrode 301 with respect to the transparent electrode307, the properties of the liquid crystal material of the respectivephase modulating element may be varied, thereby to provide a variabledelay to light incident thereon. The effect is to provide phase-onlymodulation to the wavefront, i.e. no amplitude effect occurs.

The described LCOS SLM outputs spatially modulated light in reflection.Reflective LCOS SLMs have the advantage that the signal lines, gatelines and transistors are below the mirrored surface, which results inhigh fill factors (typically greater than 90%) and high resolutions.Another advantage of using a reflective LCOS spatial light modulator isthat the liquid crystal layer can be half the thickness than would benecessary if a transmissive device were used. This greatly improves theswitching speed of the liquid crystal (a key advantage for theprojection of moving video images). However, the teachings of thepresent disclosure may equally be implemented using a transmissive LCOSSLM.

Generating Multiple Holograms from a Source Image

The following embodiments concern specific techniques which may include:(1) calculating a source image from a target image; (2) determining aprimary image from the source image; (3) determining a plurality ofsecondary images from the primary image; and (4) calculating a hologramcorresponding to each secondary image. In accordance with thesetechniques, a plurality of holograms corresponding to the target imageare calculated. In some embodiments (e.g. the target image issufficiently high resolution), the source image is the same as thetarget image. In some embodiments (e.g. warping is ignored), the primaryimage is the same as the source image. Step 1 may include upscaling.Step 3 includes sampling or sub-sampling. The upscaling and sub-samplingprocesses are different—that is, they are not the simple inverse orreverse of each other. Therefore, a desired ratio between the resolutionof the target image and the resolution of a secondary image may beobtained.

In accordance with conventional techniques, a single hologramcorresponding to a target image is calculated. The hologram is sent tothe display engine of a spatial light modulator in a data frame whichmay be a HDMI frame. The size of the hologram determined for the image(i.e. number of hologram pixels) may be less than the size of thespatial light modulator (i.e. number of SLM pixels). Thus, whendisplayed, the hologram may occupy only a part of the surface area ofthe SLM (i.e. only some of the SLM pixels). In this case, a tilingengine may be implemented for writing the hologram to the pixels of theSLM according to a tiling scheme in order to use more of the SLM pixels.

In some embodiments, a target image for projection is “upscaled” to forma source image having an increased number of pixels. Thus, theresolution (in terms of the number of pixels) is increased. Theupscaling of an image may increase the number of pixels by a power oftwo, since the number of pixels is multiplied in both the x- andy-directions. For example, an image may be upscaled by 4 in the x- andy-directions. For example, each individual pixel may be replicated in a4×4 array of pixels (i.e. with the same pixel value) in the upscaledimage. In consequence, an image comprising an n×m array of pixels is“upscaled” or “over-sampled” to obtain a 4n×4m array of pixels formingan oversampled or upscaled version of the image. Theover-sampled/upscaled image may be used as the source image as describedbelow. More complex methods of upscaling the target image may be used.

Sub-Sampling Using Checkerboarding

FIG. 4 shows an example technique for determining a pair of holograms H1and H2 from respective secondary images 1 and 2 derived from a primaryimage in accordance with embodiments. In the embodiments described inthis section of the disclosure, the primary image is the source image.The following description refers to the source image (rather than theprimary image) for simplicity.

Referring to FIG. 4, an example source image (shown at the top of thedrawing) comprising an 4×8 array of image pixels is processed (e.g. byan image processing engine) to generate a pair of secondary images 1 and2 (shown in the middle of the drawing) based on a “checkerboard” layoutor pattern. Secondary image 1 is generated using every other image pixelof the source image in a first checkerboard pattern, and filling theremaining pixels with a “zero”. Thus, secondary image 1 includes theimage pixels from the source image at locations (1, 1), (1, 3) . . . (2,2), (2, 4) . . . (3, 1), (3, 3) . . . and (4, 2) . . . (4, 8). Secondaryimage 2 is generated using the opposite or inverse image pixels of thesource image to secondary image 1. Thus, secondary image 2 is generatedusing every other image pixel of the source image in a secondcheckerboard pattern that is opposite to (i.e. the inverse of) the firstcheckerboard pattern, and filling the remaining pixels with a “zero”.Thus, secondary image 2 includes the image pixels from the source imageat locations (1, 2), (1, 4) . . . (2, 1), (2, 3) . . . (3, 2), (3, 4) .. . and (4, 1) . . . (4, 7). Each of secondary images 1 and 2 is thenprocessed (e.g. by a hologram engine) to determine a correspondinghologram H1, H2 (shown at the bottom of the drawing). Any suitablemethod may be used to calculate the hologram, such as the algorithmsdescribed above.

FIG. 5 shows holographic reconstructions produced by sequentiallydisplaying holograms H1 and H2 based on a generalisation of the exampletechnique shown in FIG. 4, in accordance with embodiments.

In particular, FIG. 5 shows a subset of image spots formed by a firstholographic reconstruction of a first hologram H1 corresponding tosecondary image 1 (shown on the left-hand side of the drawing), in afirst checkerboard pattern. FIG. 5 shows a subset of image spots formedby a second holographic reconstruction of a second hologram H2corresponding to secondary image 2 (shown in the middle of the drawing),in a second checkerboard pattern, which is the opposite or inverse ofthe first checkerboard pattern. Secondary image 1 is derived by samplingthe pixels (or groups/clusters of pixels) of a source image with thefirst checkerboard pattern (e.g. sampling odd-numbered pixels inodd-numbered rows and even-numbered pixels in even-numbered rows), andzeroing out the other (un-sampled) pixels. Secondary image 2 is derivedby sampling the pixels (or groups/clusters of pixels) of the sourceimage with the second checkerboard pattern (e.g. sampling even-numberedpixels in odd-numbered rows and odd-numbered pixels in even-numberedrows), and zeroing out the other (unsampled) pixels. FIG. 5 furthershows the combined holographic reconstruction appearing to the viewer byforming the first and second holographic reconstructions, in turn,within the integration time of the human eye (shown on the right-handside of the drawing).

By using the checkerboarding approach, the spacing between the imagespots (or “image pixels”) of each individual holographic reconstructionshown in FIG. 5, is increased by a factor or two by reducing the numberof hologram pixels in H1 and H2. It can be said that the spatialresolution of each holographic reconstruction (density of image spots inthe replay field) is reduced by a factor of two. The two holographicreconstructions can be interlaced together, in time, by using (e.g.adding) a phase-ramp or software grating function (as described above)to translate one of the holographic reconstructions relative to theother such that the image spots of one reconstruction fill the gapsbetween image spots of the other reconstruction. This is advantageousbecause it helps prevents any overlap between adjacent image spots (i.e.it reduces or prevents “pixel crosstalk”). As described above, theoverlapping of adjacent image spots or image pixels can produceinterference which appears as grain/noise to the viewer. By timeinterlacing the display of the first and second holograms H1 andH2—forming the first and second holographic reconstructions in turnrather than at the same time—this interference can be minimised.

In embodiments, each of the holograms H1 and H2 may be sequentiallywritten to, and thus displayed on, the SLM at a speed that issufficiently fast that the corresponding holographic reconstructions areformed within the integration time of the human eye. Thus, a viewer,observing the replay field on which the holographic reconstructions areformed, sees a single projected image rather than a dynamically changingprojected image corresponding to multiple holographic reconstructionsformed one after the other. The projected image therefore appears to bea faithful and complete reconstruction of the source image.

Alternatively, the holograms H1 and H2 may be written to, and thusdisplayed on, two different respective SLMs, at substantially the sametime, in an arrangement that enables the corresponding holographicreconstructions to be formed in a common area of the holographic replayplane, for example by providing different respective optical paths fromeach SLM, towards the holographic replay plane. Thus, a viewer,observing the replay field on which the holographic reconstructions areformed, sees a single projected image rather than two separate projectedimages corresponding to multiple holographic reconstructions formed fromdifferent respective SLMs. The projected image therefore appears to be afaithful and complete reconstruction of the source image.

As the skilled person will appreciate, whilst FIGS. 4 and 5 showgenerating two secondary images from the source image, it is possible togenerate three or more secondary images and calculate correspondingholograms. This can be achieved using “checkerboarding” by increasingthe spacing (number of un-sampled pixels) between the sampled imagepixels (or groups/clusters of pixels) of the source image, therebyincreasing the number of checkerboard patterns. For example, threecheckerboard patterns may be used (each checkerboard pattern samplingevery third pixel in each row) to generate three secondary images fromthe source image, and so on.

The checkerboarding approach described above can be used together withany suitable technique for generating a plurality of secondary imagesfrom a primary image. Examples of such techniques are provided below.

Sub-Sampling with Kernels

FIGS. 6 to 9 illustrate a technique for generating secondary imagesusing so-called “kernels”. In particular, a kernel is used to directlysample (or “sub-sample”) pixels of a high-resolution image to derive aplurality of secondary images. In the embodiments described in thissection of the disclosure, each secondary image comprises fewer pixelsthan the source image. However, other embodiments are contemplated, inwhich sampling is used to produce secondary images that have the samenumber of pixels as the source image, or even more pixels than thesource image. In the embodiments described in this section of thedisclosure, the primary image is also the same as the source image. Thefollowing description refers to the source image (rather than theprimary image) for simplicity.

FIG. 6 shows an example of a high-resolution source image, which may bean upscaled “target image” for projection by a holographic projector, asdescribed below with reference to FIGS. 16A and 16B. In particular,source image 600 comprises an n×m array of pixels P comprising n rowsand m columns. The number of pixels in the array has a higher resolutionthan the desired resolution of the image (holographic reconstruction)projected by the holographic projector. For example, the source imagemay have a minimum of 2x the desired resolution, such as 4 x or 8 x thedesired resolution. In this way, when sampling (sub-sampling) isperformed, as described below, the holographic reproduction of thesource image has the desired resolution even though the resolution isreduced compared to the high-resolution source image. Thus, it may besaid that the target image is “over-sampled” or “upscaled” to producethe source image, and that source image is then “sampled” or“sub-sampled” to achieve the desired net resolution of the image(holographic reconstruction). In accordance with FIGS. 6 to 9, samplingis performed using kernels.

FIG. 7A shows an example technique for sampling the source image of FIG.6 to derive a first secondary image and FIG. 7B shows an exampletechnique for sampling the source image of FIG. 6 to derive a secondsecondary image, that is diagonally offset to the first secondary image.

Referring to FIG. 7A, the source image 700 is sampled using a so-called“kernel” to derive a first secondary image 750A. A kernel may beconsidered as a moving sampling window. FIG. 8 show a generic kernel 800and an example, specific kernel 800′. In the illustrated examples, thekernel comprises a sampling window for a 4×4 array of pixels (group ofpixels). According to the presently-disclosed methods, the kernel actsas a sampling window for generating one or more secondary images, fromthe pixels of the source image. For each secondary image that is to begenerated, the kernel is incrementally moved to a series of samplingwindow positions that overlay contiguous (i.e. adjacent andnon-overlapping) 4×4 arrays/groups of pixels of the source image. It maybe said that the kernel operates on contiguous 4×4 arrays of pixels ofthe source image. For each contiguous sampling window position, thekernel operates to determine a single sub-sampled pixel value A for thesecondary image 750A that is representative of the 16 pixels values P ofthe source image that are within the 4×4 sampling window, at its currentposition. There is a correspondence between the sampling windowposition, within the source image, which gives rise to a particularpixel value, and the position of the pixel to which that pixel value isassigned, within the secondary image.

By way of non-limiting example, FIG. 7A (top of drawing) shows thepixels P₁₁ to P₄₄ of the source image sampled by the kernel at a firstsampling window position 710A, covering a 4×4 array of pixels startingat pixel P₁₁ (top left of source image). The 16 pixels in that array, atthat first sampling window position, are used to derive the pixel valueA₁₁ in the upper left corner of the sub-sampled image (secondary image,bottom of drawing). In this example, there are 12 contiguous samplingwindow positions on the source image.

The kernel may determine the pixel value A, for a pixel of the secondaryimage, at a sampling window position based on the pixel values P of eachof the 16 pixels in the 4×4 pixel array of the source image weightedaccording to a kernel weight K for the respective pixel, as describedbelow. Thus, the kernel operates at each sampling window position so asto determine a plurality of corresponding pixel values and therebyderive a sampled (e.g. sub-sampled) secondary image 750A. In FIG. 7A,the sampled image 750A comprises 12 pixels arranged in a 3×4 array ofpixels A, where each pixel value A corresponds to one of the 12contiguous sampling window positions on the source image.

FIG. 8 shows an example kernel 800′ comprising a sampling window for a4×4 pixel array, which may be used in FIG. 7A.

Kernel 800 is a generic kernel for a 4×4 pixel array sampling windowsize (i.e. 4×4 pixel array kernel size). Kernel 800 comprises a 4×4array of kernel pixels, each kernel pixel defining a weight K for apixel value P of a corresponding pixel of the 4×4 pixels of the sourceimage in the sampling window. At each sampling window position, thesub-sampled pixel value A (for the secondary image) may be determined asan average of the kernel-weighted pixel values P (from the sourceimage). Thus, kernel 800 defines kernel weights K₁₁ to K₄₄ correspondingto pixel values P₁₁ to P₄₄ of the source image at the first samplingwindow position, and pixel value A₁₁ of the under-sampled image isdetermined as 1/16×((K₁₁×P₁₁)+(K₂₂×P₂₂)+ . . . (K₄₄×P₄₄)).

Kernel 800′ shows an example of the generic kernel 800, which defineskernel weights K for pixel kernels in an example embodiment. Inparticular, the weight of kernel pixels in the centre of the kernel is“3”, whilst the weight of kernel pixels at the periphery of the kernelis “1”. Thus, pixels values P of inner sampled pixels of the sourceimage (i.e. pixels at the centre of the sampling window) have higherweight than pixel values P of outer samples pixels of the source image.As the skilled person will appreciate, many variations of the values ofkernel weights are possible according to application requirements. Inaddition, any kernel shape and size (arrangement, aspect ratio andnumber of kernel pixels) corresponding to sampling window may be chosenaccording to application requirements. For example, the kernel weightscan be selected to achieve the optimal antialiasing results.

Referring to FIG. 7B, the source image 700 is sampled, using the samekernel as the sampling in FIG. 7A described above, to derive a secondsecondary image 750B. Thus, contiguous 4×4 pixel arrays of the sourceimage 700 are under-sampled at contiguous sampling window positions.However, the sampling window positions used to derive the secondsecondary image 750B are diagonally offset from, but partially overlap,the sampling window positions used in FIG. 7A to derive the firstsecondary image 750A. In particular, in FIG. 7B, the first samplingwindow position 710B is diagonally offset by 2×2 pixel positions (i.e.two pixels in each direction), so that it overlaps the lower rightquadrant of 2×2 pixel positions of the first sampling window position710A of FIG. 7A. Thus, FIG. 7B (top of drawing) shows the pixels P₃₃ toP₆₆ of the source image sampled by the kernel at a first sampling windowposition 710B starting at pixel P₃₃ (top left of source image offset by2×2 pixels), to derive the pixel value B₁₁ of the sub-sampled image(secondary image) comprising pixels B₁₁ to B₃₄ (bottom of drawing).

Thus, a plurality of secondary images corresponding to a source image isgenerated by sampling the source image using a sampling scheme (kernelcomprising a 4×4 array kernel pixels). Each secondary image may comprisefewer pixels than the source image. Each pixel of each secondary imagemay comprise a contribution from (e.g. may comprise a weighted averageof the pixel values of) a plurality of pixels, within the source image.Furthermore, in the example shown in FIGS. 7A and 7B, each secondaryimage has the same number of pixels (3×4 pixel array) as each of therespective other secondary images. A hologram is determined for each ofthe plurality of secondary images, and each hologram is displayed on adisplay device to form a holographic reconstruction corresponding toeach secondary image on a replay plane. The holograms may be displayedin turn by a common device (i.e. by the same device), or they may bedisplayed substantially simultaneously by two different respectivedisplay devices or on two respectively different zones or areas of thesame display device.

As described above, when the holograms are displayed in turn, each ofthe plurality of holograms is displayed, in turn, on the display devicewithin the integration time of the human eye, so that the holographicreconstructions thereof on the replay plane are “interlaced” and appearas a faithful and complete reconstruction of the source/target image.

In order to increase the resolution of the perceived holographicreconstruction on the replay field, the holographic reconstruction ofthe second hologram is spatially displaced on the replay plane relativeto the holographic reconstruction of the first hologram. In particular,the spatial displacement between the holographic reconstructions formedby displaying the first and second holograms comprises a diagonaloffset, so that the image spots of the second holographic reconstructionfill in the gaps between the image spots of the first holographicreconstruction. This technique is referred to herein as “diagonalinterlacing”. In some embodiments, this is achieved by adding aphase-ramp (also referred to above as a grating function) to at leastone of the holograms in order to spatially displace the correspondingreplay field on the replay plane.

FIG. 9 shows the combined/integrated holographic reconstruction on thereplay plane, as seen by a viewer, formed by displaying the first andsecond holograms, in turn, using diagonal interlacing. In particular,FIG. 9 comprises image spots of a first holographic reconstructionformed by displaying the first hologram derived from the first secondaryimage, shown as empty circles, and image spots of a second holographicreconstruction formed by displaying the second hologram derived from thesecond secondary image, shown as hatched circles. The image spots of thesecond holographic reconstruction are spatially displaced on the replayplane relative to the image spots of the first holographicreconstruction, by a diagonal offset represented by arrow X. It may besaid that the first and second holographic reconstructions arediagonally offset with respect to each other. In particular, the secondholographic reconstruction is spatially displaced relative to the firstholographic reconstruction in a diagonal direction (e.g. 45 degrees) andby distance such that the image spots of the second holographicreconstruction fill in the gaps between the image spots of the firstholographic reconstruction. For example, as illustrated in FIG. 9, eachimage spot of the second holographic reconstruction is positionedcentrally between up to 4 image spots of the first holographicreconstruction. Thus, the combined/integrated holographic reconstructionon the replay plane has an increased pixel density (number of imagespots in the replay field).

As the skilled person will appreciate, the diagonal displacement of thefirst and second holographic reconstructions may be achieved bycontrolling the display device to change the position of the replayfield. This may be achieve using known techniques for changing theposition of the replay field on the replay plane (e.g. using x and yphase-ramps), sometimes referred to as “beam steering”. The amount ofthe displacement in each direction is chosen to correspond with thedisplacement between the first and second secondary images.

Accordingly, a simple technique is provided for “diagonal interlacing”of a plurality of holographic reconstructions corresponding to a sourceimage, wherein each holographic reconstruction has a checkboard patternof image spots. Each holographic reconstruction has fewer image spots,and thus a lower image spot density/resolution, than a singleholographic reconstruction corresponding to the source image, and isdisplayed at a different time and/or by a different display device or bya different zone or area within a common display device. This reducesthe problem of interpixel interference and pixel crosstalk. Furthermore,since the image spots of the respective holographic reconstructions arediagonally displaced, by moving the replay field on the replay plane, sothat the image spots of one holographic reconstructions fills the gapsin the checkerboard pattern between the image spots of anotherholographic reconstruction, the combined/integrated holographicreconstruction has a higher density of image spots, and thus a higherresolution than either/any of the individual holographicreconstructions.

Sub-Sampling an Intermediate Image with Warping Correction

FIGS. 10 to 15 illustrate an alternative technique for generatingsecondary images. This technique samples (or “sub-samples”) pixels of aprimary image derived from a high-resolution source image thatcompensates for so-called “warping”. The sampling process derives aplurality of secondary images, so that each may secondary image comprisefewer pixels than the primary image. For the avoidance of doubt, in theembodiments described in this section of the disclosure, the primaryimage is not the same as the source image. The primary image is anintermediate image derived from the source image in accordance with awarping map or pair of warping maps (e.g. x and y).

FIG. 10A shows an example source image comprising 16 pixels. The sourceimage may be an upscaled version of the target image for projection, asdescribed below with reference to FIGS. 16A and 16B. In somesystems—such as head-up display—an image (e.g. virtual image) of theholographic reconstruction is formed. In the example of head-up display,an image of the holographic reconstruction may be viewed from aso-called eye-box which is a region in space within which the image maybe seen. The image of the replay field may be formed by an opticalsystem, such as an optical relay system, which may include optics havingoptical power and/or an image combiner. The image formed by the opticalsystem may be distorted. The distortion may be modelled by consideringthe individual displacement (x and y) of each pixel. In practicalapplications such as a head-up display, such distortions may be causedby magnifying optics, freeform optics, windscreens and the like in theoptical path from the replay plane to the eye-box. This effect is knownas “warping”.

Conventionally, image pre-processing is used to compensate for theeffects of warping. In particular, the source image ispre-distorted—using e.g. an anti-warping map—to compensate for theknown/measured warping effect. Thus, a pre-processed version of thesource image is projected, wherein the pre-processed image or“anti-warped image” effectively includes distortions (e.g. displacedpixels) having the inverse effect to the warping effect.

FIG. 10B shows an example of an intermediate image in accordance withthe present disclosure, wherein the intermediate image is formed bywarping the source image. As shown in FIG. 10B, the position of each ofthe 16 image pixels is translated in the x-y plane, as compared to theirrespective positions in the source image in FIG. 10A. The positions ofthe pixels in the intermediate image may be determined by establishingthe translation in the x- and y-directions caused by warping. This canbe determined by computational ray tracing from the eye-box back to thereplay plane, or by using a camera to measure real-world displacementsat the eye-box and interpolating the results (as described in moredetail below in relation to the warping maps).

In the embodiments described in this section of the disclosure, thewarped image (i.e. an intermediate image not the source or target image)as illustrated in FIG. 10B is sampled (e.g. sub-sampled) to generate aplurality of secondary images. As in the prior technique, the samplingprocess for generating first and second secondary images includes adiagonal offset with a partial overlap, as described below. Thesecondary images used for calculating the holograms effectivelycompensate for the warping effect that an optical system would otherwisehave on the source image, because the secondary images are calculatedfrom the intermediate image (i.e. the warped image) not the sourceimage. Accordingly, as well as the advantages of “diagonal interlacing”as described above, this technique has the additional advantage ofsimultaneously compensating for warping caused by an optical relaysystem arranged to image the holographic reconstructions on the replayplane.

Referring to FIG. 11A, the warped image of FIG. 10B is sampled using agroup of four symmetrically arranged circular sampling windows, which,in the illustrated arrangement, overlay the entire warped image todetermine the pixel values of a first secondary image. In this example,each secondary image has only 2×2 pixels but the person skilled in theart will appreciate that the method can be scaled-up to any number ofpixels. Each sampling window corresponds to a single pixel of thesub-sampled image. Thus, the sampling in this example reduces the numberof pixels from 16 (in the source and intermediate images) to 4 (in eachsecondary image). FIG. 11B shows the first secondary image comprisingfour pixels having pixel values C₁₁, C₁₂, C₂₁, C₂₂ derived from thesampling shown in FIG. 11A. FIG. 12 shows a magnified view of the firstsampling window, corresponding to top-left circle of FIG. 11A. The firstsampling window samples a group of five unequally-spaced pixels havingpixel values P₁, P₂, P₅, P₆ and P₉. As shown in FIG. 11 A, the othersampling windows sample groups of a different number of unequally spacedpixels. In the illustrated example, it is assumed that the intensity ofeach spot has a Gaussian distribution. Accordingly, pixel values P of“inner pixels” (i.e. located near the centre of the sampling window)have a high weighting (e.g. “5”), and pixels values P of “outer pixels”(i.e. located near the edge of the sampling window) have a low weighting(e.g. “1”), according to a Gaussian distribution. Thus, a singlesub-sampled pixel value C₁₁ of the secondary image, that isrepresentative of the pixels of the warped image within the firstsampling window, can be calculated. For example, the pixel value C₁₁ maybe calculated as a function of the pixel values P₁, P₂, P₅, P₆ and P₉,such as the sum or average of the weighted pixel values P₁, P₂, P₅, P₆and P₉. In some embodiments, the weighting technique assumes that theintensity of each pixel decreases from the centre of the sampling windowin accordance with a Gaussian distribution. The technique may thereforeinclude measuring the distance of each pixel from the centre of thesampling window and weighting the value assigned to that pixel (like thekernel method) based on the distance. For example, the grey level ofeach pixel may be multiplied a factor representative of the distance(again, based on a Gaussian distribution). Pixels may be included withina sampling window if the distance is less than a threshold value. Somepixels may be included in the calculation of more than one pixel of thesecondary image. Some pixels may be included in the calculation of apixel for a first secondary image and a second secondary image. Thesampling windows shown are circular but other shapes may be employed.

FIG. 13A shows the sampling of the warped image of FIG. 10B to determinea second secondary image. In particular, the warped image is sub-sampledusing the same group of four symmetrically arranged circular samplingwindows (shown as circles) as in FIG. 11A. However, the position of thegroup of sampling windows used to derive the second secondary image isdiagonally offset from, but partially overlaps, the position of thegroup of sampling windows used in FIG. 11A to derive the first secondaryimage. FIG. 13B shows the second secondary image comprising four pixelvalues D₁₁, D₁₂, D₂₁, D₂₂ derived from the sampling shown in FIG. 13A.Each pixel value D of the second secondary image is determined byassuming a Gaussian distribution, in the same way as the pixel values Cof the first secondary image are determined, as described above withreference to FIGS. 11A and 12.

Warping Maps

FIG. 14 shows an example displacement map that may be used to determinea warped, intermediate image (e.g. FIG. 10B) corresponding to a sourceimage (e.g. FIG. 10A). In particular, the map corresponds to the area ofthe replay field and the dots correspond to specific locations on thereplay field. Downward arrows represent a negative displacement of thepixel due to warping, upward arrows represent a positive displacement ofthe pixel due to warping and the length of each arrow represents themagnitude of the displacement. FIG. 14 shows the displacement caused bywarping in one direction (e.g. displacement in the x-direction). It willbe appreciated that another displacement map is needed for the otherdirection (e.g. displacement in the y-direction). Accordingly, eachdisplacement map may be used to determine the magnitude and direction ofthe displacement of pixels (in the x- and y-directions) based on theirrespective locations within the source/target image to derive a warpedimage. As described above, displacement maps can be determined by raytracing and the like. In some embodiments, the displacement map iscreated by projecting an array of dots and using a camera to measure thedisplacement of each dot in the image formed by the optical system. Insome embodiments, this process includes placing a screen showing theun-warped array of dots on the image plane (e.g. virtual image plane) ofthe optical system and using a camera to measure the actual displacementof each light dot from the position of the corresponding dot on thescreen in order to provide the plurality of data points plotted in FIG.14. The variable phase-ramp (software grating) function may be used aspart of the measurement process—e.g. by determining the gradient of thephase-ramp function required to move a dot to the correct position. Thereader will appreciate that such as process requires accuratepositioning and calibration. A detailed description of the process isbeyond the scope of this disclosure but, nevertheless, within thecapabilities of the person skilled in the art. The techniques disclosedherein require warping maps and it is not relevant to the techniques ofthe present disclosure how those warping maps are formed. The personskilled in the art will appreciate that it is common in imaging to beprovided with warping maps for image correction.

It will be appreciated that the displacements measurements in FIG. 14provide information related to only specific points on the replay plane.In some embodiments, interpolation is used to derive a complete warpingmap from FIG. 14. Accordingly, all pixels—e.g. P1 to P16 of FIG. 10A—canbe mapped to respective warped positions.

FIG. 15 shows an example complete warping map that may be used todetermine a warped intermediate image corresponding to a source image.In particular, the map shows a surface corresponding to the area of thereplay field and the surface coordinates (up/down or the z-direction)correspond to the direction and magnitude of displacement caused bywarping at the position. FIG. 15 shows the displacement caused bywarping in one direction (e.g. displacement in the x-direction). It willbe appreciated that another displacement map is needed for the otherdirection (e.g. displacement in the y-direction). Accordingly, eachwarping map may be used to determine the magnitude and direction of thedisplacement of pixels (in the x- and y-directions) based on theirrespective locations within the source/target image to derive a warpedimage which may be used as the basis of a method to compensate forwarping as disclosed herein. As described above, warping maps can bedetermined by real-world measurements and interpolation.

As the reader will appreciate, a plurality of pairs (x and y) of warpingmaps may be provided for a corresponding plurality of eye-box positions(e.g. to accommodate tall or short viewers, different viewing positionsetc). Accordingly, implementations may select one of a plurality ofwarping maps for use in sub-sampling a primary image in response toeye-tracking data.

Thus, a plurality of secondary images is generated by sampling (e.g.sub-sampling) the intermediate image using a sampling scheme (circularsampling windows). Each secondary image in this example comprises fewerpixels than the intermediate image, however other examples arecontemplated in which each secondary image has the same number of pixelsas, or more pixels than, an intermediate image. Furthermore, as shown inFIGS. 11B and 13B, each secondary image in this example has the samenumber of pixels (2×2 pixel array) as each of the respective othersecondary images. A hologram is determined for each of the plurality ofsecondary images, and each hologram is displayed, on a display device toform a holographic reconstruction corresponding to each secondary imageon a replay plane. Each hologram may be displayed in turn on a displaydevice. Alternatively, or additionally, two or more display devices (ortwo or more zones or areas within the same display device) may beprovided, to display two or more respective holograms concurrently.

When each of the plurality of holograms is displayed, in turn, on thedisplay device, they are displayed within the integration time of thehuman eye, so that the holographic reconstructions thereof on the replayplane are “diagonally interlaced” and appear as a faithful and completereconstruction of the source image. Accordingly, the holographicreconstruction of a second hologram corresponding to a second secondaryimage is spatially displaced on the replay plane relative to theholographic reconstruction of a first hologram corresponding to a firstsecondary image. In particular, the spatial displacement between theholographic reconstructions formed by displaying the first and secondholograms comprises a diagonal offset. This may be achieved as describedabove. The amount of the displacement in each direction is chosen tocorrespond with the displacement between the first and second secondaryimages.

When each of the plurality of holograms is displayed on a differentrespective SLM (or within a different respective zone or area of thesame SLM), at substantially the same time, the corresponding holographicreconstructions may be substantially overlapping. That is, theholographic reconstructions may be formed in a common area of the replayplane at substantially the same time and may be “diagonally interlaced”as detailed above. Thus, a viewer, observing the replay field on whichthe holographic reconstructions are formed, sees a single projectedimage rather than two separate projected images corresponding tomultiple holographic reconstructions formed from different respectiveSLMs. The projected image therefore appears to be a faithful andcomplete reconstruction of the source image. Accordingly, there aredisclosed herein techniques for “diagonal interlacing” of a plurality ofholographic reconstructions corresponding to a source image, optionally,whilst compensating for warping by sub-sampling a warped version of thesource image (i.e. an intermediate image). Owing to interlacing, eachholographic reconstruction has fewer image spots, and thus a lower imagespot density/resolution, than a single holographic reconstructioncorresponding to the entire source image. This reduces the problem ofinterpixel interference and pixel crosstalk. Since the image spots ofthe respective holographic reconstructions are diagonally displaced, bymoving the replay field on the replay plane, so that the image spots ofone holographic reconstructions fills the gaps between the image spotsof another holographic reconstruction, the perceived resolution of thedevice is not reduced by the interlacing process.

Accordingly, there is disclosed herein a method of holographicprojection. The method receives a source image for projection, whereinthe source image comprising pixels. The method generates a plurality ofsecondary images from the source image, wherein each secondary image maycomprise fewer pixels than the source image. Each secondary image isgenerated by sampling a primary image, the primary image comprising oneof: the source image and an intermediate image. The method furthercalculates a hologram corresponding to each secondary image to form aplurality of holograms. The method displays each hologram on a displaydevice such as an SLM. Each hologram may be displayed in turn on adisplay device. Alternatively, or additionally, two or more displaydevices (or two or more areas or zones, with a common display device)may be provided, to display two or more respective hologramsconcurrently. The method Illuminates each hologram during display toform a holographic reconstruction corresponding to each secondary imageon a replay plane.

Since the holographic reconstruction of a smaller hologram has fewerimage spots in the same replay field size, the density of image spots,and thus the image resolution, is lower than for a larger hologram.Moreover, the signal-to-noise ratio (SNR) may be higher if more tiles ofthe smaller hologram are displayed in accordance with the chosen tilingscheme to improve pixel uniformity.

In consequence of these and other differences between smaller and largerholograms, it may be appropriate to use a different refresh rate forsmaller holograms compared to larger holograms. For example, a part ofthe source image for which a smaller hologram (with lower resolution andpotentially higher SNR depending on the chosen tiling scheme) isgenerated, could be refreshed at a higher speed or sub-frame rate than apart of the source image for which a larger hologram (with higherresolution and lower SNR) is generated. For instance, in a head-updisplay (HUD) application, for example for use in a moving vehicle, itmay be desirable to display objects in the “near field” (appearingcloser to the viewer) at a relatively low resolution but a relativelyhigh refresh rate, whilst displaying objects in the “far field”(appearing further away to the viewer) at a relatively high resolutionbut at a relatively low refresh rate, or vice versa. As the skilledperson will appreciate, other variations are possible in accordance withthe present disclosure.

In some embodiments, there is provided a display device such as ahead-up display comprising the holographic projector and an opticalrelay system. The optical relay system is arranged to form a virtualimage of each holographic reconstruction. In some embodiments, thetarget image comprises near-field image content in a first region of thetarget image and far-field image content in a second region of thetarget image. A virtual image of the holographically reconstructednear-field content is formed a first virtual image distance from aviewing plane, e.g. eye-box, and a virtual image of the holographicallyreconstructed far-field content is formed a second virtual imagedistance from the viewing plane, wherein the second virtual imagedistance is greater than the first virtual image distance. In someembodiments, one hologram of the plurality of holograms corresponds toimage content of the target image that will be displayed to a user inthe near-field (e.g. speed information) and another hologram of theplurality of holograms corresponds to image content of the target imagethat will be projected into the far-field (e.g. landmark indicators ornavigation indicators). The image content for the far-field may berefreshed more frequently than the image content for the near-field, orvice versa.

The approach disclosed herein provides multiple degrees of freedom, andthus a more flexible holographic projector. For example, the techniquedefining how the secondary images are derived from the source image maybe dynamically varied. In particular, the primary image may bedynamically-changed in response to eye-tracking data by providing awarping map or warping map pair for a plurality of eye-box positions. Anintermediate image (i.e. warped image) may be formed in real-time usingthe warping map/s. In the embodiments described in the next section, theimage processing engine may dynamically change the scheme used to derivethe secondary images from the source image, based on applicationrequirements and/or external factors indicated by a control signal suchas eye-tracking data. In addition, different tiling schemes may be used.A display engine (or tiling engine thereof) may dynamically change thetiling scheme used to display a hologram according to applicationrequirements and/or external factors indicated by a control signal. Thisflexibility is highly valuable in a real-world projector, which maydisplay different source images in a dynamically varying environment.For example, a holographic projector may be situated in a movingvehicle.

Sampling Based on Warping Map

FIGS. 16 to 18 illustrate a technique for generating secondary images inaccordance with another example. This technique also -samples (or“sub-samples”) pixels of a high-resolution image (e.g. “upscaled” or“over-sampled” image), which forms a primary image. The sub-samplingprocess derives a plurality of secondary images from the source image,so that each secondary image may comprise fewer pixels than thesource/primary image. For the avoidance of doubt, in the embodimentsdescribed in this section of the disclosure, the primary image is thesame as the source image. The term “upscaled image” is used in thissection of the disclosure to refer to the source/primary image.

Notably, in these embodiments, the sampling window positions used forsub-sampling groups of pixels of the primary image are determined basedon the warping map/s.

FIG. 16A shows an example target image 1600 comprising 16 pixels. Thetarget image 1600 comprises a 4×4 array of pixels having pixel “1” to“16”. FIG. 16B shows an over-sampled version 1610 of target image 1600(herein “upscaled image”). Upscaled image 1610 has been over-sampled (orupscaled) by a factor of four in both the x- and y-directions. Inparticular, each pixel of the target image is repeated or replicated ina 4×4 array (herein “block”) of the identical pixel value in upscaledimage 1610. Thus, upscaled image 1610 comprises 64 pixels in a 16×16array of pixels. The 16×16 array of pixels comprises 16 blocks ofidentical pixels, the pixels in each block replicating a respectivepixel of source image 1600.

As described above, warping (image distortion due to displacement ofpixels) may occur due to components of an optical relay system, whichimages the holographic reconstruction formed on the replay plane (e.g. adiffuser) to an eye-box region for a viewer. A displacement or warpingmap, as illustrated in FIGS. 14 and 15, may be used to determine thedisplacement of pixels caused by warping.

Accordingly, since the displacement of a pixel in the x- andy-directions caused by the warping effect is known, the displacedpositions can be utilized for the purpose of sub-sampling to compensatefor the warping effect. Accordingly, this technique samples (orsub-samples) groups of pixels (e.g. blocks comprising 4×4 pixel arrays)of the upscaled (source) image 1610 at displaced pixel positions tocompensate for the warping effect.

Referring to FIG. 17A, the upscaled image 1610 is sub-sampled to derivea plurality of secondary images, using the checkerboarding approach, asdescribed above. In particular, a first set of eight blocks (4×4 pixelarrays) arranged in a first checkerboard pattern (dark shading indrawing) are selected to be sub-sampled to derive a first secondaryimage, and a second set of eight pixel blocks (4×4 pixel arrays)arranged in a second checkerboard pattern (light shading with dashedoutlines in drawing), which is the inverse or opposite to the firstcheckerboard pattern, are selected to be sub-sampled to derive a secondsecondary image. FIG. 17B shows the displacement of each of the firstset of pixel blocks according to the first checkerboard pattern, asdetermined by a displacement/warping map. Similarly, FIG. 17C shows thedisplacement of each of the second set of pixel blocks according to thesecond checkerboard pattern, as determined by the displacement/warpingmap. In each case, a pixel block (4×4 pixel array) is translated in thex- and y-directions by a defined amount. For example, thedisplacement/warping map may be used to define a set of coordinates inthe warped image that can be used to determine a sampling position (e.g.a starting position of the sampling windows for image sampling). Thus,for example, the warping maps may indicate that the block of pixels “1”in the upscaled image 1610 will be displaced in the x- and y-directions(down and to the right in the drawing) by the optical relay system.

The upscaled image 1610 is sampled using a sampling window for a 4×4array of pixels at a set of sampling window positions corresponding tothe displaced positions shown in FIGS. 17B and 17C. Sampling at thedisplaced positions in FIG. 17B gives rise to a first secondary imageand sampling at the displaced positions in FIG. 17C gives rise to asecond secondary image. Thus, a single sampled pixel value for eachpixel of each secondary image is derived from the pixel values of thepixels of the upscaled image 1610 falling within the correspondingsampling window. For example, a single sub-sampled pixel value may bedetermined as an (unweighted) average of the pixel values of the pixelsof the upscaled image 1610 contained within the sampling window at thedisplaced position.

As the skilled person will appreciate, in contrast to the othertechniques disclosed herein in which the sampling window positions maybe predefined, the sampling window positions are calculated based on adisplacement/warping map for the optical relay system. The warping map/smay be a function of eye-box position and so the sampling windowpositions may be dynamically-variable.

Accordingly, using the sampling scheme shown in FIGS. 17B and 17B, firstand second secondary images are generated, each comprising 4×2 arrays ofpixels, corresponding to the source image. The first and secondsecondary images automatically compensate for the warping effect, byvirtue of the sampling scheme (i.e. the positions of the samplingwindows). As described herein, first and second holograms H1 and H2 arecalculated using the first and second secondary images. The first andsecond holograms H1 and H2 may be displayed on a spatial lightmodulator, in turn, within the integration time of the human eye, by“diagonal interlacing”. Alternatively, they may be displayedsubstantially simultaneously on two different respective SLM's, or ontwo different areas or zones within a common SLM, and irradiated so asto produce their respective holographic reconstructions on a common areaof the replay field, substantially overlapping with one another. Thus, afaithful reconstruction of the target image appears in the eye boxregion of a holographic projector, since the integrated holographicreconstruction formed by diagonal interlacing of the holographicreconstructions on the replay plane is compensated for a warping effectof the optical relay system (i.e. from the replay plane to the eye boxregion).

Although in the example described above relating to FIGS. 17A to 17C, acheckerboarding approach is used to derive secondary images, with theconventional checkerboarding sampling positions being warped by awarping map; it is also possible to apply a similar technique to akernel-based approach for deriving secondary images. That is, a warpingmap may be applied to a plurality of ‘un-warped’ contiguous samplingwindow positions—such as those that are adopted by a kernel in theexample of FIGS. 6 to 9 herein—in order to derive a ‘warped’ set ofsampling window positions, for use in deriving secondary images thathave a built-in warp correction, and therefore account for the imagewarping that an optical system may otherwise cause, to a source image.

FIG. 18 shows an example of a part of upscaled image 1610 overlaid witha sampling window 1800 or ‘kernel’. This upscaled image 1610 is anupscaled version of a 2×2 target image (or ‘original source image’, notitself shown), which has been over-sampled (or upscaled) by a factor offour in both the x- and y-directions. As described previously, thesampling window 1800 captures a block (4×4 array) of pixels of theupscaled image 1610. FIG. 18 shows the sampling window 1800 at a first‘warped’ sampling window position, which corresponds to the displacementof the first block of pixels “1” (and therefore corresponds to thedisplacement of a first ‘un-warped’ sampling window position) in theupscaled image 1610, calculated using the displacement/warping map.Thus, the sampling window 1800 overlays pixels of three adjacent blocksof pixels “2”, “3” and “4” in addition to pixels of the block of pixels“1”. In the illustrated example, using sampling window 1800 at thisfirst warped sampling window position, the first pixel value for thesecondary image is determined as the mean average of the pixel values ofthe pixels of the upscaled image 1610 that are contained within thesampling window (i.e. 1/16th of the sum of the values of the 16 pixelsor 1/16 ((3×1)+(1×2)+(9×3)+(3×4))). The other pixel values for thesecondary image can be calculated from the pixels that fall within thesampling window at each of the other warped sampling window positions.

In this example, the upscaled image 1610 is an upscaled version of a 2×2target image (not itself shown), which has been over-sampled (orupscaled) by a factor of four in both the x- and y-directions, and thesampling window or ‘kernel’ is a 4×4 array, which produces a singlepixel value of a secondary image, for every sampling window positionthat it occupies. Therefore, the sampling by the kernel effectivelydownscales the upscaled image by a factor of four in both the x- andy-directions. As a result, a single pixel of the sampled (i.e.secondary) image corresponds to a single pixel of the original sourceimage 1600 (i.e. prior to over-sampling/upscaling). Thus, the resolutionof original source image 1600 is maintained. It will be appreciated thatthis is just one example, and that different scaling may be applied forthe upscaling of an original source image and/or that a different sizeor shape of sampling window may be used, in order to achieve a desirednet ratio between the resolution of the original source image and theresolution of a secondary image that is derived therefrom.

In addition, implementations may be optimised for more efficientconsumption of memory and processing resources than other techniques forcompensating for warping. In particular, since the over-samplingtechnique to derive the upscaled image replicates pixels of the inputimage, the individual pixel values of the upscaled image need not bestored in memory. For example, consider an input image of 1024×1024pixels that is over sampled to derive an upscaled image of 4096×4096pixels. Storing the upscaled image in memory would undesirably increasethe memory usage 16 fold. However, instead of storing the upscaledimage, it is possible to create a simple memory-efficient addressingscheme. In particular, each pixel of the upscaled image will have 16possible addresses, 4 in X and 4 in Y (corresponding to the 4×4 pixelarray). Thus, an addressing scheme based on the two most significantbits of a four-bit mapping scheme can be used to identify each block or4×4 array of identical pixels in the upscaled image. Accordingly, memoryresource usage is minimised by using a binary mapping scheme for pixels,which may be used in the sub-sampling process. In addition, thetechnique of sampling a high-resolution image that has been over-sampledto a power of two, such as to the power of four as described herein,involves simple calculations using binary arithmetic. For example,addition of pixel values of 16 pixels, (contained in sampling window fora 4×4 pixel array), involves straightforward binary processing that canbe performed quickly and efficiently, for example using a binary adder.Likewise, determining an average of the pixel values of 16 pixels withina sampling window also involves straightforward binary processing thatcan be performed quickly and efficiently, for example by discarding thefour least significant bits.

As the skilled person will appreciate, many variations and modificationsmay be made to the above techniques for sub-sampling a primary image togenerate a plurality of secondary images. For example, whilst thesampling window positions having a diagonal offset are described for thepurpose of diagonal interlacing, a directional offset in only onedirection (e.g. x or y direction) may be used.

In all of the embodiments described herein, the size and shape of akernel, and/or of a sampling window, can differ from the specificexamples which have been shown and described. A kernel, or a samplingwindow, does not have to comprise a regular geometric shape, nor does ithave to have a size or a shape that resembles the configuration of thepixels in a primary/source image, which it samples, nor does it have tohave a size or a shape that resembles the configuration of the pixels ina generated secondary image. For example, circular kernels/samplingwindows, as shown in FIGS. 11A, 12 and 13A herein, may be used in any ofthe other respective embodiments. Any suitable mathematical rule ormapping scheme may be applied, to associate one or more pixels (or pixelpositions) with a kernel or sampling window, in a given position. Akernel or sampling window may, at least in some arrangements, onlypartially overlay or encompass an image (or part of an image) or a pixelthat it is sampling.

System Diagram

FIG. 19 is a schematic showing a holographic system in accordance withembodiments. A spatial light modulator (SLM) 940 is arranged to displayholograms received from a controller 930. In operation, a light source910 illuminates the hologram displayed on SLM 940 and a holographicreconstruction is formed in a replay field on a replay plane 925.Controller 930 receives one or more images from an image source 920. Forexample, image source 920 may be an image capture device such as a stillcamera arranged to capture a single still image or video camera arrangedto capture a video sequence of moving images.

Controller 930 comprises image processing engine 950, hologram engine960, data frame generator 980 and display engine 990. Image processingengine 950 receives a source image from image source 920. Imageprocessing engine 950 includes a secondary image generator 955 arrangedto generate a plurality of secondary images from a primary image basedon the source image in accordance with a defined scheme, as describedherein. Image processing engine 950 may receive a control signal orotherwise determine the scheme for generating the secondary images.Thus, each secondary image may comprise fewer pixels than the sourceimage. Image processing engine 950 may generate the plurality ofsecondary images using the source image as the primary image. The sourceimage may be upscaled version of the target image, or the imageprocessing engine may perform upscaling as described herein.Alternatively, image processing engine 950 may process the source imageto determine an intermediate image, and use the intermediate image asthe primary image. For example, the intermediate image may be an “warpedimage”, as described herein. The warped image may be determined using adisplacement map that comprises a displacement value for each pixel ofthe source image (e.g. in the x- and y-directions) representing theimage distortion caused by an optical relay system arranged to image ofeach holographic reconstruction. Image processing engine 950 maygenerate the plurality of secondary images by sampling the primaryimage, as described herein. Image processing engine 950 may determine afirst secondary image and a second secondary image, wherein the pixelvalue of each pixel of a first secondary image is calculated from afirst group of pixels of the primary image and the pixel value of eachpixel of a second secondary image is calculated from a second group ofpixels of the primary image. In some implementations, the samplingwindow used to select the second group of pixels is offset from, and/orpartially overlaps, the sampling window used to select the first groupof pixels. In other implementations, the sampling window positions, ineach case, may be arranged in a checkerboard pattern, where differentcheckerboard patterns are used for each secondary image. In someimplementations, the sampling window positions for selecting the firstand second groups of pixels are determined using a displacement map.Image processing engine 950 passes the plurality of secondary images tohologram engine 960.

Hologram engine 960 is arranged to determine a hologram corresponding toeach secondary image, as described herein. Hologram engine 960 passesthe plurality of holograms to data frame generator 980. Data framegenerator 980 is arranged to generate a data frame (e.g. HDMI frame)comprising the plurality of holograms, as described herein. Inparticular, data frame generator 980 generates a data frame comprisinghologram data for each of the plurality of holograms, and pointersindicating the start of each hologram. Data frame generator 980 passesthe data frame to display engine 990. Display engine 990 is arranged todisplay each of the plurality of holograms, on SLM 940. The hologramsmay be displayed in turn and/or the SLM 940 may in fact comprise two ormore SLM's, for displaying two or more respective hologramssubstantially concurrently, and/or two or more holograms may bedisplayed substantially concurrently on two or more distinct areas orzones of the SLM 940. Display engine 990 comprises hologram extractor992, tiling engine 970 and software optics 994. Display engine 990extracts each hologram from the data frame using hologram extractor 992and tiles the hologram according to a tiling scheme generated by tilingengine 970, as described herein. In particular, tiling engine 970 mayreceive a control signal to determine the tiling scheme, or mayotherwise determine a tiling scheme for tiling based on the hologram.Display engine 990 may optionally add a phase ramp function (softwaregrating function also called a software lens) using software optics 994,to translate the position of the replay field on the replay plane, asdescribed herein. Accordingly, for each hologram, display engine 990 isarranged to output a drive signal to SLM 940 to display each hologram ofthe plurality of holograms, according to a corresponding tiling scheme,as described herein.

Controller 930 may dynamically control how secondary image generator 955generates secondary images, as described herein. Controller 930 maydynamically control the refresh rate for holograms. As described herein,the refresh rate may be considered as the frequency at which a hologramis recalculated by hologram engine, from a next source image in asequence received by image processing engine 950 from image source 920.As described herein, dynamically controllable features and parametersmay be determined based on external factors indicated by a controlsignal. Controller 930 may receive control signals relating to suchexternal factors, or may include modules for determining such externalfactors and generating such control signals, accordingly.

As the skilled person will appreciate, the above-described features ofcontroller 930 may be implemented in software, firmware or hardware, andany combination thereof.

Additional Features

Embodiments refer to an electrically-activated LCOS spatial lightmodulator by way of example only. The teachings of the presentdisclosure may equally be implemented on any spatial light modulatorcapable of displaying a computer-generated hologram in accordance withthe present disclosure such as any electrically-activated SLMs,optically-activated SLM, digital micromirror device ormicroelectromechanical device, for example.

In some embodiments, the light source is a laser such as a laser diode.In some embodiments, the light receiving surface is a diffuser surfaceor screen such as a diffuser. The holographic projection system of thepresent disclosure may be used to provide an improved head-up display(HUD) or head-mounted display. In some embodiments, there is provided avehicle comprising the holographic projection system installed in thevehicle to provide a HUD. The vehicle may be an automotive vehicle suchas a car, truck, van, lorry, motorcycle, train, airplane, boat, or ship.

The quality of the holographic reconstruction may be affect by theso-called zero order problem which is a consequence of the diffractivenature of using a pixelated spatial light modulator. Such zero-orderlight can be regarded as “noise” and includes for example specularlyreflected light, and other unwanted light from the SLM.

In the example of Fourier holography, this “noise” is focused at thefocal point of the Fourier lens leading to a bright spot at the centreof the holographic reconstruction. The zero order light may be simplyblocked out however this would mean replacing the bright spot with adark spot. Some embodiments include an angularly selective filter toremove only the collimated rays of the zero order. Embodiments alsoinclude the method of managing the zero-order described in Europeanpatent 2,030,072, which is hereby incorporated in its entirety byreference.

The size of the holographic replay field (i.e. the physical or spatialextent of the holographic reconstruction) is determined by the pixelspacing of the spatial light modulator (i.e. the distance betweenadjacent light-modulating elements, or pixels, of the spatial lightmodulator). The smallest feature which may be formed on the replay fieldmay be called a “resolution element”, “image spot” or an “image pixel”.Typically, each pixel of the spatial light modulator has a quadrangularshape. The Fourier transform of a quadrangular aperture is a sincfunction and therefore each image pixel is a sinc function. Morespecifically, the spatial intensity distribution of each image pixel onthe replay field is a sinc function. Each sinc function may beconsidered as comprising a peak-intensity primary diffractive order anda series of decreasing-intensity higher diffractive orders extendingradially away from the primary order. The size of each sinc function(i.e the physical or spatial extent of each sinc function) is determinedby the size of the spatial light modulator (i.e. the physical or spatialextent of the aperture formed by the array of light-modulating elementsor spatial light modulator pixels). Specifically, the larger theaperture formed by the array of light-modulating pixels, the smaller theimage pixels. It is usually desirable to have small image pixels.

In some embodiments, the technique of “tiling” is implemented toincrease image quality. Specifically, some embodiments implement thetechnique of tiling to minimise the size of the image pixels whilstmaximising the amount of signal content going into the holographicreconstruction.

In some embodiments, the holographic pattern written to the spatiallight modulator comprises at least one whole tile (that is, the completehologram) and at least one fraction of a tile (that is, a continuoussubset of pixels of the hologram).

The holographic reconstruction is created within the zeroth or primarydiffraction order of the overall window defined by the spatial lightmodulator. It is preferred that the first and subsequent orders aredisplaced far enough so as not to overlap with the image and so thatthey may be blocked using a spatial filter.

In embodiments, the holographic reconstruction is colour. In examplesdisclosed herein, three different colour light sources and threecorresponding SLMs are used to provide composite colour. These examplesmay be referred to as spatially-separated colour, “SSC”. In a variationencompassed by the present disclosure, the different holograms for eachcolour are displayed on different area of the same SLM and thencombining to form the composite colour image. However, the skilledperson will understand that at least some of the devices and methods ofthe present disclosure are equally applicable to other methods ofproviding composite colour holographic images.

One of these methods is known as Frame Sequential Colour, “FSC”. In anexample FSC system, three lasers are used (red, green and blue) and eachlaser is fired in succession at a single SLM to produce each frame ofthe video. The colours are cycled (red, green, blue, red, green, blue,etc.) at a fast enough rate such that a human viewer sees apolychromatic image from a combination of the images formed by threelasers. Each hologram is therefore colour specific. For example, in avideo at 25 frames per second, the first frame would be produced byfiring the red laser for 1/75th of a second, then the green laser wouldbe fired for 1/75th of a second, and finally the blue laser would befired for 1/75th of a second. The next frame is then produced, startingwith the red laser, and so on.

An advantage of FSC method is that the whole SLM is used for eachcolour. This means that the quality of the three colour images producedwill not be compromised because all pixels of the SLM are used for eachof the colour images. However, a disadvantage of the FSC method is thatthe overall image produced will not be as bright as a correspondingimage produced by the SSC method by a factor of about 3, because eachlaser is only used for a third of the time. This drawback couldpotentially be addressed by overdriving the lasers, or by using morepowerful lasers, but this would require more power to be used, wouldinvolve higher costs and would make the system less compact.

An advantage of the SSC method is that the image is brighter due to allthree lasers being fired at the same time. However, if due to spacelimitations it is required to use only one SLM, the surface area of theSLM can be divided into three parts, acting in effect as three separateSLMs. The drawback of this is that the quality of each single-colourimage is decreased, due to the decrease of SLM surface area availablefor each monochromatic image. The quality of the polychromatic image istherefore decreased accordingly. The decrease of SLM surface areaavailable means that fewer pixels on the SLM can be used, thus reducingthe quality of the image. The quality of the image is reduced becauseits resolution is reduced. Embodiments utilise the improved SSCtechnique disclosed in British patent 2,496,108 which is herebyincorporated in its entirety by reference.

Some embodiments describe 2D holographic reconstructions by way ofexample only. In other embodiments, the holographic reconstruction is a3D holographic reconstruction. That is, in some embodiments, eachcomputer-generated hologram forms a 3D holographic reconstruction.

The methods and processes described herein may be embodied on acomputer-readable medium. The term “computer-readable medium” includes amedium arranged to store data temporarily or permanently such asrandom-access memory (RAM), read-only memory (ROM), buffer memory, flashmemory, and cache memory. The term “computer-readable medium” shall alsobe taken to include any medium, or combination of multiple media, thatis capable of storing instructions for execution by a machine such thatthe instructions, when executed by one or more processors, cause themachine to perform any one or more of the methodologies describedherein, in whole or in part.

The term “computer-readable medium” also encompasses cloud-based storagesystems. The term “computer-readable medium” includes, but is notlimited to, one or more tangible and non-transitory data repositories(e.g., data volumes) in the example form of a solid-state memory chip,an optical disc, a magnetic disc, or any suitable combination thereof.In some example embodiments, the instructions for execution may becommunicated by a carrier medium. Examples of such a carrier mediuminclude a transient medium (e.g., a propagating signal that communicatesinstructions).

It will be apparent to those skilled in the art that variousmodifications and variations can be made without departing from thescope of the appended claims. The present disclosure covers allmodifications and variations within the scope of the appended claims andtheir equivalents.

1-30. (canceled)
 31. A holographic projector arranged to project atarget image, the holographic projector comprising: an image processingengine arranged to generate a plurality of secondary images by samplinga primary image derived from the target image; a hologram enginearranged to determine a hologram corresponding to each secondary imageto form a plurality of holograms; a display engine arranged to displayeach hologram on a display device, and a light source arranged toIlluminate each hologram during display to form a holographicreconstruction corresponding to each secondary image on a replay plane.32. A holographic projector as claimed in claim 31 wherein the displayengine is arranged to display each hologram in turn on the device; orwherein the display device is arranged to display each hologramsubstantially simultaneously on different respective areas of thedisplay device; or wherein a plurality of display devices is providedand wherein the display engine is arranged to display each hologramsubstantially simultaneously on different respective display devices,within the plurality of display devices.
 33. A holographic projector asclaimed in claim 31 wherein the secondary images each have fewer pixelsthan the primary image.
 34. A holographic projector as claimed in claim31 wherein each secondary image comprises a plurality of pixels,calculated from corresponding groups of pixels of the primary image at aplurality of positions of a sampling window; and wherein each pixelvalue of each secondary image is calculated from a corresponding groupthat comprises a plurality of pixels of the primary image that fallwithin the sampling window at a respective one of the plurality ofsampling window positions.
 35. A holographic projector as claimed inclaim 31 wherein the sampling comprises calculating the pixel value ofeach pixel of a secondary image from a respective group of pixels of theprimary image falling within a sampling window such that there is apositional correspondence between each pixel of the secondary image andthe respective group of pixels of the primary image and/or therespective sampling window.
 36. A holographic projector as claimed inclaim 35 wherein the pixel value of each pixel of a first secondaryimage is calculated from a first group of pixels of the primary imagefalling within the sampling window at a first set of sampling windowpositions and the pixel value of each pixel of a second secondary imageis calculated from a second group of pixels of the primary image fallingwithin the sampling window at a second set of sampling window positions.37. A holographic projector as claimed in claim 31 wherein the imageprocessing engine is arranged to process the source image using adisplacement map to form an intermediate image as the primary image,wherein the displacement map comprises a displacement value for eachpixel of the source image representing the image distortion caused by anoptical relay system arranged to form an image of each holographicreconstruction.
 38. A holographic projector as claimed in claim 31wherein the plurality of secondary images comprises a first secondaryimage and a second secondary image, wherein the pixel values of thepixels of the first secondary image are calculated from a first set ofpixel blocks of the primary image falling within the sampling window ata first set of sampling window positions, and the pixel values of thepixels of the second secondary image are calculated from a second set ofpixel blocks of the primary image falling within the sampling window ata second set of sampling window positions; wherein the first set ofpixel blocks of the primary image are arranged in a first checkerboardpattern, and the second set of pixel blocks of the primary image arearranged in a second checkerboard pattern that is opposite to the firstcheckerboard pattern.
 39. A holographic projector as claimed in claim 31wherein the number of secondary images is greater than two.
 40. Aholographic projector as claimed in claim 31 wherein each hologram ofthe plurality of holograms is displayed at a speed, and/or at a positionrelative to the respective other holograms, such that the holographicreconstructions thereof are formed within the integration time of thehuman eye.
 41. A method of holographically-projecting a reconstructionof a target image, the method comprising: generating a plurality ofsecondary images by sampling a primary image derived from the targetimage; calculating a hologram corresponding to each secondary image toform a plurality of holograms; displaying each hologram on a displaydevice, and illuminating each hologram during display to form aholographic reconstruction corresponding to each secondary image on areplay plane.
 42. The method of claim 41 wherein the step of displayingeach hologram on a display device comprises any one or more of:displaying each hologram in turn on the display device; or displayingeach hologram substantially simultaneously on different respective areasof the display device; or displaying each hologram substantiallysimultaneously on different respective display devices, within aplurality of display devices.
 43. The method of claim 41 wherein thegenerated secondary images each have fewer pixels than the primaryimage.
 44. The method of claim 41 wherein the step of generating aplurality of secondary images comprises, for each secondary image,calculating a plurality of pixels, from corresponding groups of pixelsof the primary image at a plurality of positions of a sampling window;and wherein each pixel value of each secondary image is calculated froma corresponding group that comprises a plurality of pixels of theprimary image that fall within the sampling window at a respective oneof the plurality of sampling window positions.
 45. A method of claim 41further comprising: receiving the target image for projection; andupscaling the target image to form a source image having more pixelsthan the target image, wherein the primary image is derived from thesource image.
 46. A method as claimed in claim 45 wherein upscalingcomprises repeating each pixel value of the target image in respectivecontiguous group of pixels of the source image, wherein there is apositional correspondence between each pixel of the target image and thecorresponding group of pixels of the source image having the same pixelvalue.
 47. A method as claimed in claim 45 wherein the primary image isthe source image.
 48. A method as claimed in claim 45 further comprisingprocessing the source image using a displacement map to form anintermediate image as the primary image, wherein the displacement mapcomprises a displacement value for each pixel of the source imagerepresenting the image distortion caused by an optical relay systemarranged to form an image of each holographic reconstruction.
 49. Amethod as claimed in claim 41 wherein each pixel value of the secondaryimage is calculated by individually weighting the pixel values of therespective group of pixels of the primary image falling within therespective sampling window.
 50. A method as claimed in claim 49comprising: calculating the pixel value of each pixel of a firstsecondary image from a respective first group of pixels of the primaryimage falling within the sampling window at a first set of samplingwindow positions; and calculating the pixel value of each pixel of asecond secondary image from a respective second group of pixels of theprimary image falling within the sampling window at a second set ofsampling window positions, wherein the second set of sampling windowpositions is diagonally offset from the first set of sampling windowpositions and/or each sampling window at the second set of samplingwindow positions partially overlaps the corresponding sampling window atthe first set of sampling window positions.